[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:57 dist-test-slave-1zqn krb5kdc[7859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:57 dist-test-slave-1zqn krb5kdc[7859](info): set up 2 sockets
Aug 09 04:44:57 dist-test-slave-1zqn krb5kdc[7859](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 04:44:58.757922 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.008s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:44:58 dist-test-slave-1zqn krb5kdc[7859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:44:58.897717 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:37471
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:37471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:59.013737 7874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:59.014168 7874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:59.014273 7874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:59.018414 7874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:59.018499 7874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:59.018535 7874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:59.018592 7874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:59.018630 7874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:59.024155 7874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:37471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:37471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:59.025213 7874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:59.026268 7874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:59.035669 7879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.035782 7882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.035689 7880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:59.036180 7874 server_base.cc:1034] running on GCE node
I20260809 04:44:59.036763 7874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:59.037837 7874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:59.038997 7874 hybrid_clock.cc:648] HybridClock initialized: now 1786250699038987 us; error 91 us; skew 500 ppm
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250699, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:59.041989 7874 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:44:59.042927 7874 webserver.cc:460] Webserver started at http://127.7.171.62:44709/ using document root <none> and password file <none>
I20260809 04:44:59.043677 7874 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:59.043745 7874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:59.043992 7874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:59.046152 7874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "9f500a6d22ce4ad9b2361ab2440fc5c3"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "a6065986bcd4bf6bda46cd092add1029"
server_key_iv: "8e3fe7ebc38727c215df0c1f65cd9c82"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.046742 7874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "9f500a6d22ce4ad9b2361ab2440fc5c3"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "a6065986bcd4bf6bda46cd092add1029"
server_key_iv: "8e3fe7ebc38727c215df0c1f65cd9c82"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.050662 7874 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:44:59.053289 7890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.054631 7874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:44:59.054764 7874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "9f500a6d22ce4ad9b2361ab2440fc5c3"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "a6065986bcd4bf6bda46cd092add1029"
server_key_iv: "8e3fe7ebc38727c215df0c1f65cd9c82"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.054889 7874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:59.076967 7874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:59.080513 7874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:59.080741 7874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:59.088946 7874 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:37471
I20260809 04:44:59.088958 7952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:37471 every 8 connection(s)
I20260809 04:44:59.089960 7874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:59.092911 7953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:59.095432 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 7874
I20260809 04:44:59.095633 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:44:59.096992 7852 external_mini_cluster.cc:1428] Setting key 8c2c73ac96fe9541f06ce72300f73a03
I20260809 04:44:59.098738 7953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3: Bootstrap starting.
I20260809 04:44:59.101913 7953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:59.102734 7953 log.cc:824] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:59.104880 7953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3: No bootstrap required, opened a new log
I20260809 04:44:59.108791 7953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f500a6d22ce4ad9b2361ab2440fc5c3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 37471 } }
I20260809 04:44:59.109007 7953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:59.109067 7953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f500a6d22ce4ad9b2361ab2440fc5c3, State: Initialized, Role: FOLLOWER
I20260809 04:44:59.109479 7953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f500a6d22ce4ad9b2361ab2440fc5c3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 37471 } }
I20260809 04:44:59.109596 7953 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:59.109684 7953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:59.109808 7953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:59.110690 7953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f500a6d22ce4ad9b2361ab2440fc5c3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 37471 } }
I20260809 04:44:59.111034 7953 leader_election.cc:302] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f500a6d22ce4ad9b2361ab2440fc5c3; no voters:
I20260809 04:44:59.111378 7953 leader_election.cc:288] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:59.111449 7958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:59.111667 7958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [term 1 LEADER]: Becoming Leader. State: Replica: 9f500a6d22ce4ad9b2361ab2440fc5c3, State: Running, Role: LEADER
I20260809 04:44:59.111964 7958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f500a6d22ce4ad9b2361ab2440fc5c3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 37471 } }
I20260809 04:44:59.112691 7953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:59.113168 7958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f500a6d22ce4ad9b2361ab2440fc5c3. Latest consensus state: current_term: 1 leader_uuid: "9f500a6d22ce4ad9b2361ab2440fc5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f500a6d22ce4ad9b2361ab2440fc5c3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 37471 } } }
I20260809 04:44:59.113353 7958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:59.113636 7960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f500a6d22ce4ad9b2361ab2440fc5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f500a6d22ce4ad9b2361ab2440fc5c3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 37471 } } }
I20260809 04:44:59.113807 7960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:59.113826 7964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:44:59.118144 7964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:59.123636 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.100340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58426 (local address 127.7.171.62:37471)
0809 04:44:59.100803 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:59.100824 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:59.102353 (+ 1529us) server_negotiation.cc:407] Connection header received
0809 04:44:59.106032 (+ 3679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:59.106050 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:59.106351 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:59.106692 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:59.108895 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.110275 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.111641 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.112401 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.112591 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:59.117921 (+ 5330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:59.117927 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:59.117939 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:59.117970 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:59.119990 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.120721 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.120727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.120732 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.120874 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.121246 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.121250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.121251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.121651 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:59.121896 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:59.122458 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 04:44:59.122749 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":147,"threads_started":1}
I20260809 04:44:59.125280 7964 catalog_manager.cc:1347] Generated new cluster ID: 3d8fd12013e544dab6a58c5bebf71dcd
I20260809 04:44:59.125344 7964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:59.147241 7964 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:59.148346 7964 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:59.153721 7964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f500a6d22ce4ad9b2361ab2440fc5c3: Generated new TSK 0
I20260809 04:44:59.154663 7964 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:44:59.192281 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:37471
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:59.309659 7980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:59.309926 7980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:59.309969 7980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:59.314234 7980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:59.314301 7980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:59.314396 7980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:44:59.319871 7980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:37471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:59.321055 7980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:59.322155 7980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:44:59.332459 7980 server_base.cc:1034] running on GCE node
W20260809 04:44:59.332376 7985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.332660 7988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.332373 7986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:59.333125 7980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:59.333783 7980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:59.334995 7980 hybrid_clock.cc:648] HybridClock initialized: now 1786250699334977 us; error 27 us; skew 500 ppm
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250699, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:59.337822 7980 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:44:59.338732 7980 webserver.cc:460] Webserver started at http://127.7.171.1:38197/ using document root <none> and password file <none>
I20260809 04:44:59.339331 7980 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:59.339452 7980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:59.339682 7980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:59.341758 7980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "8cdacaa01a0d4b30b771e002d23da37d"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "1de83310b7385483f6a5ba4154042372"
server_key_iv: "a2b32c3dfd1e474d8dbd88f72274ca2b"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.342347 7980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "8cdacaa01a0d4b30b771e002d23da37d"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "1de83310b7385483f6a5ba4154042372"
server_key_iv: "a2b32c3dfd1e474d8dbd88f72274ca2b"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.346217 7980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 04:44:59.348742 7997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.350047 7980 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:59.350185 7980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "8cdacaa01a0d4b30b771e002d23da37d"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "1de83310b7385483f6a5ba4154042372"
server_key_iv: "a2b32c3dfd1e474d8dbd88f72274ca2b"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.350323 7980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:59.374007 7980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:59.377579 7980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:59.377815 7980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:59.378489 7980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:59.379421 7980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:59.379498 7980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.379580 7980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:59.379624 7980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.389873 7980 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:36169
I20260809 04:44:59.389873 8124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:36169 every 8 connection(s)
I20260809 04:44:59.390847 7980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:59.400301 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 7980
I20260809 04:44:59.400429 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:59.400713 7852 external_mini_cluster.cc:1428] Setting key 37c2193a9d127ea9dc8f906b7e2e0958
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250699, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:44:59.406325 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.392949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:50791 (local address 127.7.171.62:37471)
0809 04:44:59.393117 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:59.393121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:59.393951 (+ 830us) server_negotiation.cc:407] Connection header received
0809 04:44:59.395376 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:59.395380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:59.395440 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:59.395547 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:59.396871 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.397426 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.398582 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.398841 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.398962 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:59.401739 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:59.401742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:59.401744 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:59.401769 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:59.403740 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.404355 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.404363 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.404368 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.404478 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.404849 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.404856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.404861 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.405169 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:59.405276 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:59.406065 (+ 789us) server_negotiation.cc:299] Negotiation successful
0809 04:44:59.406191 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:44:59.407487 8128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.393323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:37471 (local address 127.7.171.1:50791)
0809 04:44:59.393800 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:59.393843 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:59.395160 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:59.395737 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:59.395744 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:59.396188 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:59.396716 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:59.396730 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.397576 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.397579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:59.398450 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:59.398459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.399008 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.399012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:59.399192 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:59.399816 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:59.399832 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:59.401539 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:59.403871 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:59.403877 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:59.403890 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:59.404249 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:59.404589 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:59.404592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:59.404593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:59.404738 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:59.405305 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:59.405314 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:59.405674 (+ 360us) client_negotiation.cc:769] Sending connection context
0809 04:44:59.405904 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 04:44:59.406321 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":115,"threads_started":1}
I20260809 04:44:59.409076 8125 heartbeater.cc:347] Connected to a master server at 127.7.171.62:37471
I20260809 04:44:59.409348 8125 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:59.409891 8125 heartbeater.cc:511] Master 127.7.171.62:37471 requested a full tablet report, sending...
I20260809 04:44:59.411708 7909 ts_manager.cc:194] Registered new tserver with Master: 8cdacaa01a0d4b30b771e002d23da37d (127.7.171.1:36169)
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:44:59.413445 7909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:50791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:44:59.431641 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:37471
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:59.549190 8131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:59.549456 8131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:59.549502 8131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:59.553699 8131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:59.553769 8131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:59.553869 8131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:44:59.559636 8131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:37471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:59.560796 8131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:59.561861 8131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:59.571871 8139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.571887 8136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.571871 8137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:59.572153 8131 server_base.cc:1034] running on GCE node
I20260809 04:44:59.572665 8131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:59.573280 8131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:59.574467 8131 hybrid_clock.cc:648] HybridClock initialized: now 1786250699574432 us; error 46 us; skew 500 ppm
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250699, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:59.577469 8131 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:44:59.578397 8131 webserver.cc:460] Webserver started at http://127.7.171.2:43001/ using document root <none> and password file <none>
I20260809 04:44:59.579032 8131 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:59.579136 8131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:59.579377 8131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:59.581532 8131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "b927342f743842318910ade05838c3a5"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "d5953e52673efd103c7e2a09e05bb7bd"
server_key_iv: "ffbcb8dc382e85526b1db1908a0771b1"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.582139 8131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "b927342f743842318910ade05838c3a5"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "d5953e52673efd103c7e2a09e05bb7bd"
server_key_iv: "ffbcb8dc382e85526b1db1908a0771b1"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.586169 8131 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:59.588810 8146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.590090 8131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:59.590236 8131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "b927342f743842318910ade05838c3a5"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "d5953e52673efd103c7e2a09e05bb7bd"
server_key_iv: "ffbcb8dc382e85526b1db1908a0771b1"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.590356 8131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:59.619514 8131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:59.623046 8131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:59.623284 8131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:59.624120 8131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:59.624984 8131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:59.625034 8131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.625083 8131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:59.625099 8131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.634625 8131 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:40657
I20260809 04:44:59.634657 8272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:40657 every 8 connection(s)
I20260809 04:44:59.635557 8131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:59.639567 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 8131
I20260809 04:44:59.639670 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:59.639969 7852 external_mini_cluster.cc:1428] Setting key ffbf14784d14d73a16540023ca719d97
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250699, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:44:59.651023 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.637466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:32859 (local address 127.7.171.62:37471)
0809 04:44:59.637614 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:59.637618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:59.638492 (+ 874us) server_negotiation.cc:407] Connection header received
0809 04:44:59.639933 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:59.639937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:59.639999 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:59.640164 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:59.641546 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.642290 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.643458 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.643813 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.643899 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:59.646828 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:59.646836 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:59.646843 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:59.646884 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:59.648775 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.649308 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.649314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.649319 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.649422 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.649781 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.649787 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.649790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.650060 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:59.650176 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:59.650679 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 04:44:59.650816 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:44:59.651839 8275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.637858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:37471 (local address 127.7.171.2:32859)
0809 04:44:59.638319 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:59.638366 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:59.639699 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:59.640327 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:59.640334 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:59.640824 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:59.641389 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:59.641408 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.642415 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.642418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:59.643331 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:59.643340 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.643979 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.643985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:59.644245 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:59.644904 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:59.644931 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:59.646611 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:59.648898 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:59.648903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:59.648916 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:59.649194 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:59.649530 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:59.649532 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:59.649534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:59.649671 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:59.650174 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:59.650179 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:59.650419 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 04:44:59.650585 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 04:44:59.650846 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":104,"threads_started":1}
I20260809 04:44:59.653333 8273 heartbeater.cc:347] Connected to a master server at 127.7.171.62:37471
I20260809 04:44:59.653667 8273 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:59.654277 8273 heartbeater.cc:511] Master 127.7.171.62:37471 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
I20260809 04:44:59.655525 7909 ts_manager.cc:194] Registered new tserver with Master: b927342f743842318910ade05838c3a5 (127.7.171.2:40657)
I20260809 04:44:59.656203 7909 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:32859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:44:59.673331 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:37471
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:59.789073 8278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:59.789362 8278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:59.789443 8278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:59.793586 8278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:59.793679 8278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:59.793773 8278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:44:59.799256 8278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:37471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:44:59.800493 8278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:59.801586 8278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:59.811686 8285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.811672 8284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.811672 8288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:59.811975 8278 server_base.cc:1034] running on GCE node
I20260809 04:44:59.812475 8278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:59.813158 8278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:59.814337 8278 hybrid_clock.cc:648] HybridClock initialized: now 1786250699814303 us; error 46 us; skew 500 ppm
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250699, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:59.817409 8278 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:44:59.818452 8278 webserver.cc:460] Webserver started at http://127.7.171.3:42113/ using document root <none> and password file <none>
I20260809 04:44:59.819089 8278 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:59.819157 8278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:59.819397 8278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:59.821702 8278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "e937be5c4aaf4c4dbd8a987c245d1202"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "8b0d1ba04a9fdb3aaed5c1bb55b4adbc"
server_key_iv: "93fee0e82480fe7506a3a7556109ec23"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.822317 8278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "e937be5c4aaf4c4dbd8a987c245d1202"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "8b0d1ba04a9fdb3aaed5c1bb55b4adbc"
server_key_iv: "93fee0e82480fe7506a3a7556109ec23"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.826412 8278 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:59.829182 8298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.830591 8278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:59.830729 8278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "e937be5c4aaf4c4dbd8a987c245d1202"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1zqn"
server_key: "8b0d1ba04a9fdb3aaed5c1bb55b4adbc"
server_key_iv: "93fee0e82480fe7506a3a7556109ec23"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.830883 8278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:59.841917 8278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:59.845456 8278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:59.845690 8278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:59.846331 8278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:59.847196 8278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:59.847273 8278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.847384 8278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:59.847426 8278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.857369 8278 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:34757
I20260809 04:44:59.857393 8421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:34757 every 8 connection(s)
I20260809 04:44:59.858378 8278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:59.860702 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 8278
I20260809 04:44:59.860831 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:59.861150 7852 external_mini_cluster.cc:1428] Setting key a127318a60b5f11084ffeb917f9e8796
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250699, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:44:59.873214 7955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.860440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:50215 (local address 127.7.171.62:37471)
0809 04:44:59.860591 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:59.860595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:59.861712 (+ 1117us) server_negotiation.cc:407] Connection header received
0809 04:44:59.863503 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:59.863509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:59.863569 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:59.863652 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:59.864974 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.865583 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.866765 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.866975 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.867061 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:59.869682 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:59.869685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:59.869687 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:59.869710 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:59.871187 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.871649 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.871652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.871653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.871729 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.872242 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.872244 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.872246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.872422 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:59.872501 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:59.872971 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 04:44:59.873082 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:44:59.874127 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.860997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:37471 (local address 127.7.171.3:50215)
0809 04:44:59.861549 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:59.861595 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:59.863253 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:59.863853 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:59.863863 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:59.864343 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:59.864840 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:59.864854 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.865744 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.865746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:59.866652 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:59.866662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.867120 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.867123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:59.867267 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:59.867925 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:59.867942 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:59.869519 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:59.871298 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:59.871302 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:59.871313 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:59.871561 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:59.871836 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:59.871838 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:59.871840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:59.872149 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:59.872515 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:59.872520 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:59.872754 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 04:44:59.872906 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 04:44:59.873238 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":187,"threads_started":1}
I20260809 04:44:59.875344 8422 heartbeater.cc:347] Connected to a master server at 127.7.171.62:37471
I20260809 04:44:59.875628 8422 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:59.876237 8422 heartbeater.cc:511] Master 127.7.171.62:37471 requested a full tablet report, sending...
I20260809 04:44:59.877297 7908 ts_manager.cc:194] Registered new tserver with Master: e937be5c4aaf4c4dbd8a987c245d1202 (127.7.171.3:34757)
I20260809 04:44:59.877877 7908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:50215
I20260809 04:44:59.888141 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250699, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250699, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:44:59.916675 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.906278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36896 (local address 127.7.171.1:36169)
0809 04:44:59.906636 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:59.906657 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:59.906697 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:44:59.906773 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:59.906778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:59.906978 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:59.907149 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:59.907909 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.908793 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.909732 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.910122 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.910257 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:59.913051 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:59.913061 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:59.913075 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:59.913104 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:59.914777 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.915198 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.915203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.915205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.915288 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.915524 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.915526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.915527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.915783 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:59.916020 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:59.916378 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 04:44:59.916528 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":120,"threads_started":1}
W20260809 04:44:59.917759 8030 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36896
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:44:59 dist-test-slave-1zqn krb5kdc[7859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:44:59.946137 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.937210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36910 (local address 127.7.171.1:36169)
0809 04:44:59.937374 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:59.937381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:59.937406 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:44:59.937610 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:59.937613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:59.937663 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:59.937754 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:59.938565 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.939108 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.939958 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:59.940314 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:59.940459 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:59.942877 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:59.942882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:59.942885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:59.942914 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:59.944376 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.944863 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.944867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.944869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.944956 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:59.945243 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:59.945247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:59.945249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:59.945440 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:59.945606 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:59.945840 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 04:44:59.945963 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:44:59.947252 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 7980
I20260809 04:44:59.956915 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 8131
I20260809 04:44:59.964550 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 8278
I20260809 04:44:59.970934 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 7874
2026-08-09T04:44:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2349 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:00 dist-test-slave-1zqn krb5kdc[8441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:00 dist-test-slave-1zqn krb5kdc[8441](info): set up 2 sockets
Aug 09 04:45:00 dist-test-slave-1zqn krb5kdc[8441](info): commencing operation
krb5kdc: starting...
W20260809 04:45:01.122913 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:01 dist-test-slave-1zqn krb5kdc[8441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:01.259194 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:39501
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:35647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:39501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:01.375123 8455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:01.375440 8455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:01.375530 8455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:01.379652 8455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:01.379745 8455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:01.379786 8455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:01.379845 8455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:01.379886 8455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:01.385182 8455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:39501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:39501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:01.386278 8455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:01.387327 8455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:01.396394 8463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:01.396591 8461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:01.396927 8460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:01.396970 8455 server_base.cc:1034] running on GCE node
I20260809 04:45:01.397786 8455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:01.399282 8455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:01.400465 8455 hybrid_clock.cc:648] HybridClock initialized: now 1786250701400441 us; error 35 us; skew 500 ppm
Aug 09 04:45:01 dist-test-slave-1zqn krb5kdc[8441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250701, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:01.403419 8455 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:45:01.404547 8455 webserver.cc:460] Webserver started at http://127.7.171.62:32871/ using document root <none> and password file <none>
I20260809 04:45:01.405377 8455 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:01.405462 8455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:01.405686 8455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:01.407992 8455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "25fc5221fbba4080979784181dc10603"
format_stamp: "Formatted at 2026-08-09 04:45:01 on dist-test-slave-1zqn"
server_key: "277632342acc6347dc4cb2d16ebeedfa"
server_key_iv: "ce7c357982a2b7c37d7237ee45bfc620"
server_key_version: "encryptionkey@0"
I20260809 04:45:01.408645 8455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "25fc5221fbba4080979784181dc10603"
format_stamp: "Formatted at 2026-08-09 04:45:01 on dist-test-slave-1zqn"
server_key: "277632342acc6347dc4cb2d16ebeedfa"
server_key_iv: "ce7c357982a2b7c37d7237ee45bfc620"
server_key_version: "encryptionkey@0"
I20260809 04:45:01.412823 8455 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:01.415664 8472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:01.417114 8455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:45:01.417263 8455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "25fc5221fbba4080979784181dc10603"
format_stamp: "Formatted at 2026-08-09 04:45:01 on dist-test-slave-1zqn"
server_key: "277632342acc6347dc4cb2d16ebeedfa"
server_key_iv: "ce7c357982a2b7c37d7237ee45bfc620"
server_key_version: "encryptionkey@0"
I20260809 04:45:01.417387 8455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:01.437614 8455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:01.441447 8455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:01.441711 8455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:01.449673 8455 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:39501
I20260809 04:45:01.449702 8528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:39501 every 8 connection(s)
I20260809 04:45:01.450681 8455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:01.453521 8529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:01.457126 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 8455
I20260809 04:45:01.457237 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:01.457540 7852 external_mini_cluster.cc:1428] Setting key 0d5c181e00e6496df66698fb4494c7d0
I20260809 04:45:01.459861 8529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603: Bootstrap starting.
I20260809 04:45:01.462509 8529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:01.463306 8529 log.cc:824] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:01.465449 8529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603: No bootstrap required, opened a new log
Aug 09 04:45:01 dist-test-slave-1zqn krb5kdc[8441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:01.469837 8529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25fc5221fbba4080979784181dc10603" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39501 } }
I20260809 04:45:01.470064 8529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:01.470153 8529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25fc5221fbba4080979784181dc10603, State: Initialized, Role: FOLLOWER
I20260809 04:45:01.470644 8529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25fc5221fbba4080979784181dc10603" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39501 } }
I20260809 04:45:01.470772 8529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:01.470862 8529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:01.470986 8529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:01.471947 8529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25fc5221fbba4080979784181dc10603" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39501 } }
I20260809 04:45:01.472350 8529 leader_election.cc:302] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25fc5221fbba4080979784181dc10603; no voters:
I20260809 04:45:01.472702 8529 leader_election.cc:288] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:01.472792 8535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:01.473048 8535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [term 1 LEADER]: Becoming Leader. State: Replica: 25fc5221fbba4080979784181dc10603, State: Running, Role: LEADER
I20260809 04:45:01.473232 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:01.459271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36474 (local address 127.7.171.62:39501)
0809 04:45:01.459767 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:01.459788 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:01.459825 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:45:01.460602 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:01.460621 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:01.460969 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:01.461290 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:01.462211 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:01.463337 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:01.464397 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:01.464883 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:01.465061 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:01.467952 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:01.467958 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:01.467973 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:01.468001 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:01.469884 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:01.470404 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:01.470410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:01.470414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:01.470557 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:01.470877 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:01.470889 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:01.470891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:01.471369 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:01.471614 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:01.471867 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 04:45:01.472230 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":186,"threads_started":1}
I20260809 04:45:01.473364 8535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25fc5221fbba4080979784181dc10603" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39501 } }
I20260809 04:45:01.474157 8529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:01.474620 8535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25fc5221fbba4080979784181dc10603. Latest consensus state: current_term: 1 leader_uuid: "25fc5221fbba4080979784181dc10603" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25fc5221fbba4080979784181dc10603" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39501 } } }
I20260809 04:45:01.474731 8535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:01.475045 8536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25fc5221fbba4080979784181dc10603" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25fc5221fbba4080979784181dc10603" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39501 } } }
I20260809 04:45:01.475183 8536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:01.475749 8539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:01.479084 8539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:01.485266 8539 catalog_manager.cc:1347] Generated new cluster ID: 16966d85d4fc45afa1e1a92eca4bee8f
I20260809 04:45:01.485359 8539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:01.530288 8539 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:01.531356 8539 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:01.540591 8539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25fc5221fbba4080979784181dc10603: Generated new TSK 0
I20260809 04:45:01.541392 8539 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:45:01.579463 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:39501
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:35647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:01.698597 8557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:01.698915 8557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:01.698962 8557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:01.703022 8557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:01.703088 8557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:01.703147 8557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:01.708616 8557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:39501
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:01.709697 8557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:01.710686 8557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:01.721002 8564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:01.720932 8569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:01.721024 8565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:01.721444 8557 server_base.cc:1034] running on GCE node
I20260809 04:45:01.721979 8557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:01.723064 8557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:01.724262 8557 hybrid_clock.cc:648] HybridClock initialized: now 1786250701724246 us; error 31 us; skew 500 ppm
Aug 09 04:45:01 dist-test-slave-1zqn krb5kdc[8441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250701, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:01.727214 8557 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:45:01.728163 8557 webserver.cc:460] Webserver started at http://127.7.171.1:38553/ using document root <none> and password file <none>
I20260809 04:45:01.728765 8557 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:01.728858 8557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:01.729101 8557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:01.731223 8557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "18b41b23cb4f41f29ac3132e40b7da49"
format_stamp: "Formatted at 2026-08-09 04:45:01 on dist-test-slave-1zqn"
server_key: "1c4a658a27fe364a291651211f6fe362"
server_key_iv: "63f41fc4dc5ee13c730849c4f0822ffd"
server_key_version: "encryptionkey@0"
I20260809 04:45:01.731791 8557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "18b41b23cb4f41f29ac3132e40b7da49"
format_stamp: "Formatted at 2026-08-09 04:45:01 on dist-test-slave-1zqn"
server_key: "1c4a658a27fe364a291651211f6fe362"
server_key_iv: "63f41fc4dc5ee13c730849c4f0822ffd"
server_key_version: "encryptionkey@0"
I20260809 04:45:01.735730 8557 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 04:45:01.738257 8577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:01.739502 8557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:01.739643 8557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "18b41b23cb4f41f29ac3132e40b7da49"
format_stamp: "Formatted at 2026-08-09 04:45:01 on dist-test-slave-1zqn"
server_key: "1c4a658a27fe364a291651211f6fe362"
server_key_iv: "63f41fc4dc5ee13c730849c4f0822ffd"
server_key_version: "encryptionkey@0"
I20260809 04:45:01.739758 8557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:01.775700 8557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:01.779080 8557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:01.779310 8557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:01.779961 8557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:01.780885 8557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:01.780958 8557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:01.781021 8557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:01.781080 8557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:01.790555 8557 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:42879
I20260809 04:45:01.790625 8697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:42879 every 8 connection(s)
I20260809 04:45:01.791559 8557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:01.797755 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 8557
I20260809 04:45:01.797881 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:01.798157 7852 external_mini_cluster.cc:1428] Setting key 36604fa00dd41c60033c7b0b3545c948
Aug 09 04:45:01 dist-test-slave-1zqn krb5kdc[8441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250701, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:01.806617 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:01.793559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:40889 (local address 127.7.171.62:39501)
0809 04:45:01.793729 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:01.793734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:01.794669 (+ 935us) server_negotiation.cc:407] Connection header received
0809 04:45:01.796026 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:01.796028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:01.796129 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:01.796215 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:01.797502 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:01.798311 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:01.799523 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:01.799769 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:01.799884 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:01.802822 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:01.802826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:01.802829 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:01.802863 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:01.804456 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:01.805010 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:01.805013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:01.805015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:01.805107 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:01.805468 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:01.805471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:01.805473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:01.805671 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:01.805779 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:01.806305 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 04:45:01.806445 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:45:01.807495 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:01.794032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:39501 (local address 127.7.171.1:40889)
0809 04:45:01.794500 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:01.794547 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:01.795812 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:01.796393 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:01.796400 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:01.796789 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:01.797316 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:01.797331 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:01.798433 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:01.798436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:01.799385 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:01.799395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:01.799934 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:01.799938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:01.800190 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:01.800826 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:01.800844 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:01.802636 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:01.804589 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:01.804595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:01.804612 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:01.804884 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:01.805207 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:01.805210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:01.805212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:01.805364 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:01.805779 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:01.805787 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:01.806051 (+ 264us) client_negotiation.cc:769] Sending connection context
0809 04:45:01.806264 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 04:45:01.806622 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:45:01.808928 8698 heartbeater.cc:347] Connected to a master server at 127.7.171.62:39501
I20260809 04:45:01.809195 8698 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:01.809719 8698 heartbeater.cc:511] Master 127.7.171.62:39501 requested a full tablet report, sending...
I20260809 04:45:01.811527 8491 ts_manager.cc:194] Registered new tserver with Master: 18b41b23cb4f41f29ac3132e40b7da49 (127.7.171.1:42879)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
I20260809 04:45:01.812860 8491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:40889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:45:01.832036 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:39501
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:35647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:01.953011 8703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:01.953281 8703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:01.953334 8703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:01.957433 8703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:01.957504 8703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:01.957572 8703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:01.963004 8703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:39501
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:01.964148 8703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:01.965514 8703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:01.975772 8708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:01.975790 8709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:01.975772 8712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:01.976101 8703 server_base.cc:1034] running on GCE node
I20260809 04:45:01.976747 8703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:01.977388 8703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:01.978602 8703 hybrid_clock.cc:648] HybridClock initialized: now 1786250701978585 us; error 37 us; skew 500 ppm
Aug 09 04:45:01 dist-test-slave-1zqn krb5kdc[8441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250701, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:01.981701 8703 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:45:01.982710 8703 webserver.cc:460] Webserver started at http://127.7.171.2:35351/ using document root <none> and password file <none>
I20260809 04:45:01.983332 8703 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:01.983428 8703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:01.983685 8703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:01.985898 8703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f"
format_stamp: "Formatted at 2026-08-09 04:45:01 on dist-test-slave-1zqn"
server_key: "c99156a3078b51b4e29f349e5d63c57b"
server_key_iv: "df8a0010c3e81748ea40c3a83329b3f9"
server_key_version: "encryptionkey@0"
I20260809 04:45:01.986523 8703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f"
format_stamp: "Formatted at 2026-08-09 04:45:01 on dist-test-slave-1zqn"
server_key: "c99156a3078b51b4e29f349e5d63c57b"
server_key_iv: "df8a0010c3e81748ea40c3a83329b3f9"
server_key_version: "encryptionkey@0"
I20260809 04:45:01.990497 8703 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:45:01.993111 8720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:01.994436 8703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:01.994590 8703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f"
format_stamp: "Formatted at 2026-08-09 04:45:01 on dist-test-slave-1zqn"
server_key: "c99156a3078b51b4e29f349e5d63c57b"
server_key_iv: "df8a0010c3e81748ea40c3a83329b3f9"
server_key_version: "encryptionkey@0"
I20260809 04:45:01.994720 8703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:02.010819 8703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:02.014400 8703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:02.014655 8703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:02.015316 8703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:02.016294 8703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:02.016367 8703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:02.016443 8703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:02.016495 8703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:02.027123 8703 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:46117
I20260809 04:45:02.027172 8843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:46117 every 8 connection(s)
I20260809 04:45:02.028226 8703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:02.030313 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 8703
I20260809 04:45:02.030428 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:02.030717 7852 external_mini_cluster.cc:1428] Setting key e3bb7c892da17b9ec8b51eb47749ef51
Aug 09 04:45:02 dist-test-slave-1zqn krb5kdc[8441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250701, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:45:02.043751 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.030196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:53341 (local address 127.7.171.62:39501)
0809 04:45:02.030341 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.030345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.031550 (+ 1205us) server_negotiation.cc:407] Connection header received
0809 04:45:02.033225 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.033228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.033279 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.033361 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:02.034579 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.035113 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.036448 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.036683 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.036793 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.039458 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:02.039461 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:02.039464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:02.039488 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:02.041302 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.041868 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.041872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.041874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.041958 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.042351 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.042354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.042356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.042688 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:02.042828 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.043388 (+ 560us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.043519 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
I20260809 04:45:02.044806 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.030778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:39501 (local address 127.7.171.2:53341)
0809 04:45:02.031374 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:02.031436 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:02.032999 (+ 1563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:02.033543 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:02.033551 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:02.033918 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:02.034421 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.034434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.035247 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.035251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.036321 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.036331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.036849 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.036853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.037022 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.037678 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:02.037694 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:02.039272 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:02.041441 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:02.041448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:02.041466 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:02.041757 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:02.042089 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:02.042092 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:02.042094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:02.042244 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:02.042837 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:02.042846 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:02.043116 (+ 270us) client_negotiation.cc:769] Sending connection context
0809 04:45:02.043325 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 04:45:02.043674 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":158,"threads_started":1}
I20260809 04:45:02.046192 8844 heartbeater.cc:347] Connected to a master server at 127.7.171.62:39501
I20260809 04:45:02.046478 8844 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:02.047106 8844 heartbeater.cc:511] Master 127.7.171.62:39501 requested a full tablet report, sending...
I20260809 04:45:02.048285 8491 ts_manager.cc:194] Registered new tserver with Master: e0cec089ba3341b1ba9e6f9e5b96c73f (127.7.171.2:46117)
I20260809 04:45:02.048888 8491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:53341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:45:02.063994 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:39501
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:35647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:02.181736 8850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:02.182024 8850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:02.182072 8850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:02.186128 8850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:02.186198 8850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:02.186261 8850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:02.191839 8850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:39501
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:02.193048 8850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:02.194170 8850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:02.204906 8856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:02.204890 8858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:02.204892 8855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:02.205279 8850 server_base.cc:1034] running on GCE node
I20260809 04:45:02.205762 8850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:02.206512 8850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:02.207705 8850 hybrid_clock.cc:648] HybridClock initialized: now 1786250702207680 us; error 34 us; skew 500 ppm
Aug 09 04:45:02 dist-test-slave-1zqn krb5kdc[8441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:02.211244 8850 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:45:02.212260 8850 webserver.cc:460] Webserver started at http://127.7.171.3:42493/ using document root <none> and password file <none>
I20260809 04:45:02.212843 8850 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:02.212903 8850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:02.213148 8850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:02.215266 8850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "24d020a0c08f41b3bdfe2522eb4c595b"
format_stamp: "Formatted at 2026-08-09 04:45:02 on dist-test-slave-1zqn"
server_key: "a9fa0aa54fbe7b7285924e1b21709799"
server_key_iv: "678749b38a8767b6254a150f8cc559c1"
server_key_version: "encryptionkey@0"
I20260809 04:45:02.215847 8850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "24d020a0c08f41b3bdfe2522eb4c595b"
format_stamp: "Formatted at 2026-08-09 04:45:02 on dist-test-slave-1zqn"
server_key: "a9fa0aa54fbe7b7285924e1b21709799"
server_key_iv: "678749b38a8767b6254a150f8cc559c1"
server_key_version: "encryptionkey@0"
I20260809 04:45:02.219879 8850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:45:02.222919 8866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:02.224579 8850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:02.224721 8850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "24d020a0c08f41b3bdfe2522eb4c595b"
format_stamp: "Formatted at 2026-08-09 04:45:02 on dist-test-slave-1zqn"
server_key: "a9fa0aa54fbe7b7285924e1b21709799"
server_key_iv: "678749b38a8767b6254a150f8cc559c1"
server_key_version: "encryptionkey@0"
I20260809 04:45:02.224852 8850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:02.244233 8850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:02.248142 8850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:02.248366 8850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:02.249207 8850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:02.250125 8850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:02.250175 8850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:02.250217 8850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:02.250241 8850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:02.259228 8850 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:36181
I20260809 04:45:02.259266 8989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:36181 every 8 connection(s)
I20260809 04:45:02.260262 8850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:02.262064 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 8850
I20260809 04:45:02.262192 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:02.262487 7852 external_mini_cluster.cc:1428] Setting key 83d0208f65945158afb864310b5abdb3
Aug 09 04:45:02 dist-test-slave-1zqn krb5kdc[8441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:02.275589 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.262225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:53557 (local address 127.7.171.62:39501)
0809 04:45:02.263629 (+ 1404us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.263634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.263650 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:45:02.265167 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.265169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.265215 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.265291 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:02.266752 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.267314 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.268669 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.268887 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.268970 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.271837 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:02.271840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:02.271842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:02.271866 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:02.273328 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.273837 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.273839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.273841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.273912 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.274350 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.274352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.274353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.274536 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:02.274622 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.275316 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.275458 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1310}
I20260809 04:45:02.276813 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.262646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:39501 (local address 127.7.171.3:53557)
0809 04:45:02.263153 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:02.263224 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:02.264903 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:02.265592 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:02.265600 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:02.266065 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:02.266603 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.266618 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.267469 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.267473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.268551 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.268562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.269200 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.269203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.269354 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.270130 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:02.270147 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:02.271669 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:02.273440 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:02.273445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:02.273458 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:02.273745 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:02.274021 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:02.274024 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:02.274025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:02.274251 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:02.274636 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:02.274642 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:02.274975 (+ 333us) client_negotiation.cc:769] Sending connection context
0809 04:45:02.275243 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 04:45:02.275641 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":104,"threads_started":1}
I20260809 04:45:02.278040 8990 heartbeater.cc:347] Connected to a master server at 127.7.171.62:39501
I20260809 04:45:02.278317 8990 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:02.278849 8990 heartbeater.cc:511] Master 127.7.171.62:39501 requested a full tablet report, sending...
I20260809 04:45:02.279969 8491 ts_manager.cc:194] Registered new tserver with Master: 24d020a0c08f41b3bdfe2522eb4c595b (127.7.171.3:36181)
I20260809 04:45:02.280879 8491 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:53557
I20260809 04:45:02.289053 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:45:02.300701 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.292700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36488 (local address 127.7.171.62:39501)
0809 04:45:02.292846 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.292851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.293033 (+ 182us) server_negotiation.cc:407] Connection header received
0809 04:45:02.293238 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.293241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.293293 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.293381 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:02.294126 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.294685 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.295534 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.295766 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.295877 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.297363 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:02.297366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:02.297368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:02.297391 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:02.298941 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.299408 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.299410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.299412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.299508 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.299742 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.299744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.299745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.299993 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:02.300167 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.300388 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.300495 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:45:02.305672 8491 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:02.308759 8491 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:02.324003 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.317432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:46117 (local address 127.0.0.1:43496)
0809 04:45:02.317927 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:02.317977 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:02.318116 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:02.319345 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:02.319352 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:02.319395 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:02.319646 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.319657 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.321193 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.321197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.322463 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.322475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.323505 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.323509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.323653 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.323673 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:45:02.323729 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 04:45:02.323780 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":187,"spinlock_wait_cycles":17152,"thread_start_us":90,"threads_started":1}
I20260809 04:45:02.324100 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.317635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.7.171.2:46117)
0809 04:45:02.318832 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.318845 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.318869 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:45:02.318947 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.318952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.319163 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.319499 (+ 336us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:02.319817 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.321068 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.322596 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.323385 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.323580 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.323725 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.323782 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.323907 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1084,"thread_start_us":149,"threads_started":1}
I20260809 04:45:02.324229 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.318242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39356 (local address 127.7.171.3:36181)
0809 04:45:02.318618 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.318630 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.318672 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:45:02.318756 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.318763 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.319026 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.319180 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:02.319774 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.321054 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.322656 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.323230 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.323355 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.323515 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.323903 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.324003 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":135,"threads_started":1}
I20260809 04:45:02.324314 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.318188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:42879 (local address 127.0.0.1:44780)
0809 04:45:02.318823 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:02.318836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:02.318917 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:02.319599 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:02.319602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:02.319619 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:02.319694 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.319701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.320902 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.320909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.322384 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.322395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.323486 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.323490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.323848 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.323868 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:45:02.323936 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 04:45:02.323990 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":266,"thread_start_us":102,"threads_started":1}
I20260809 04:45:02.324481 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.317633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:36181 (local address 127.0.0.1:39356)
0809 04:45:02.318404 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:02.318420 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:02.318511 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:02.319353 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:02.319356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:02.319393 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:02.319643 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.319650 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.321252 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.321257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.322548 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.322569 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.323367 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.323371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.323479 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.323515 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 04:45:02.323635 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 04:45:02.323766 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":110,"spinlock_wait_cycles":5376,"thread_start_us":103,"threads_started":1}
I20260809 04:45:02.325012 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.318480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44780 (local address 127.7.171.1:42879)
0809 04:45:02.319165 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.319172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.319193 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:45:02.319266 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.319271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.319453 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.319831 (+ 378us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:02.319934 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.320767 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.322505 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.323319 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.323601 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.323725 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.324762 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.324869 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":72,"threads_started":1}
I20260809 04:45:02.327350 8627 tablet_service.cc:1467] Processing CreateTablet for tablet df32799d8f904b36bc92735a280d4fc5 (DEFAULT_TABLE table=test-table [id=363e0737c02d4155a6c6f3e9dcc03925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:02.327442 8775 tablet_service.cc:1467] Processing CreateTablet for tablet df32799d8f904b36bc92735a280d4fc5 (DEFAULT_TABLE table=test-table [id=363e0737c02d4155a6c6f3e9dcc03925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:02.327442 8918 tablet_service.cc:1467] Processing CreateTablet for tablet df32799d8f904b36bc92735a280d4fc5 (DEFAULT_TABLE table=test-table [id=363e0737c02d4155a6c6f3e9dcc03925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:02.328405 8627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df32799d8f904b36bc92735a280d4fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:02.328476 8918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df32799d8f904b36bc92735a280d4fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:02.328476 8775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df32799d8f904b36bc92735a280d4fc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:02.334887 9007 tablet_bootstrap.cc:492] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49: Bootstrap starting.
I20260809 04:45:02.336378 9008 tablet_bootstrap.cc:492] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f: Bootstrap starting.
I20260809 04:45:02.337503 9007 tablet_bootstrap.cc:654] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:02.338238 9007 log.cc:824] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:02.338668 9008 tablet_bootstrap.cc:654] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:02.339025 9009 tablet_bootstrap.cc:492] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b: Bootstrap starting.
I20260809 04:45:02.339506 9008 log.cc:824] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:02.340425 9007 tablet_bootstrap.cc:492] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49: No bootstrap required, opened a new log
I20260809 04:45:02.340624 9007 ts_tablet_manager.cc:1397] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 04:45:02.341135 9009 tablet_bootstrap.cc:654] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:02.341406 9008 tablet_bootstrap.cc:492] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f: No bootstrap required, opened a new log
I20260809 04:45:02.341606 9008 ts_tablet_manager.cc:1397] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 04:45:02.342418 9009 log.cc:824] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:02.344543 9009 tablet_bootstrap.cc:492] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b: No bootstrap required, opened a new log
I20260809 04:45:02.344734 9009 ts_tablet_manager.cc:1397] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:45:02.344973 9008 raft_consensus.cc:348] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 46117 } } peers { permanent_uuid: "24d020a0c08f41b3bdfe2522eb4c595b" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36181 } } peers { permanent_uuid: "18b41b23cb4f41f29ac3132e40b7da49" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42879 } }
I20260809 04:45:02.345232 9008 raft_consensus.cc:374] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:02.345288 9008 raft_consensus.cc:729] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cec089ba3341b1ba9e6f9e5b96c73f, State: Initialized, Role: FOLLOWER
I20260809 04:45:02.345731 9008 consensus_queue.cc:260] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 46117 } } peers { permanent_uuid: "24d020a0c08f41b3bdfe2522eb4c595b" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36181 } } peers { permanent_uuid: "18b41b23cb4f41f29ac3132e40b7da49" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42879 } }
I20260809 04:45:02.345598 9007 raft_consensus.cc:348] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 46117 } } peers { permanent_uuid: "24d020a0c08f41b3bdfe2522eb4c595b" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36181 } } peers { permanent_uuid: "18b41b23cb4f41f29ac3132e40b7da49" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42879 } }
I20260809 04:45:02.345872 9007 raft_consensus.cc:374] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:02.345940 9007 raft_consensus.cc:729] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18b41b23cb4f41f29ac3132e40b7da49, State: Initialized, Role: FOLLOWER
I20260809 04:45:02.346403 9007 consensus_queue.cc:260] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 46117 } } peers { permanent_uuid: "24d020a0c08f41b3bdfe2522eb4c595b" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36181 } } peers { permanent_uuid: "18b41b23cb4f41f29ac3132e40b7da49" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42879 } }
I20260809 04:45:02.347133 9008 ts_tablet_manager.cc:1428] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 04:45:02.347323 9007 ts_tablet_manager.cc:1428] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 04:45:02.347525 8844 heartbeater.cc:503] Master 127.7.171.62:39501 was elected leader, sending a full tablet report...
I20260809 04:45:02.347699 8698 heartbeater.cc:503] Master 127.7.171.62:39501 was elected leader, sending a full tablet report...
I20260809 04:45:02.348606 9009 raft_consensus.cc:348] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 46117 } } peers { permanent_uuid: "24d020a0c08f41b3bdfe2522eb4c595b" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36181 } } peers { permanent_uuid: "18b41b23cb4f41f29ac3132e40b7da49" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42879 } }
I20260809 04:45:02.348883 9009 raft_consensus.cc:374] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:02.348955 9009 raft_consensus.cc:729] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d020a0c08f41b3bdfe2522eb4c595b, State: Initialized, Role: FOLLOWER
I20260809 04:45:02.349334 9009 consensus_queue.cc:260] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 46117 } } peers { permanent_uuid: "24d020a0c08f41b3bdfe2522eb4c595b" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36181 } } peers { permanent_uuid: "18b41b23cb4f41f29ac3132e40b7da49" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42879 } }
I20260809 04:45:02.350018 8990 heartbeater.cc:503] Master 127.7.171.62:39501 was elected leader, sending a full tablet report...
I20260809 04:45:02.350230 9009 ts_tablet_manager.cc:1428] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 04:45:02.465952 9014 raft_consensus.cc:482] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:02.466212 9014 raft_consensus.cc:504] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 46117 } } peers { permanent_uuid: "24d020a0c08f41b3bdfe2522eb4c595b" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36181 } } peers { permanent_uuid: "18b41b23cb4f41f29ac3132e40b7da49" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42879 } }
I20260809 04:45:02.467383 9014 leader_election.cc:288] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cec089ba3341b1ba9e6f9e5b96c73f (127.7.171.2:46117), 24d020a0c08f41b3bdfe2522eb4c595b (127.7.171.3:36181)
I20260809 04:45:02.473457 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.467968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:46497 (local address 127.7.171.2:46117)
0809 04:45:02.468147 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.468154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.468397 (+ 243us) server_negotiation.cc:407] Connection header received
0809 04:45:02.468545 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.468551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.468668 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.468766 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:02.469100 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.470028 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.471882 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.472619 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.472739 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.472812 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.473208 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.473301 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:45:02.473439 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.467809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:37481 (local address 127.7.171.3:36181)
0809 04:45:02.467940 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.467945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.468438 (+ 493us) server_negotiation.cc:407] Connection header received
0809 04:45:02.468672 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.468675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.468737 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.468819 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:02.469134 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.469998 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.471882 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.472611 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.472743 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.472812 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.473208 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.473298 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 04:45:02.473469 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.467714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:36181 (local address 127.7.171.1:37481)
0809 04:45:02.468305 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:02.468319 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:02.468523 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:02.468905 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:02.468909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:02.468923 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:02.469016 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.469025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.470169 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.470172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.471731 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.471740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.472757 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.472762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.472953 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.473039 (+ 86us) client_negotiation.cc:769] Sending connection context
0809 04:45:02.473130 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 04:45:02.473258 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":473,"thread_start_us":139,"threads_started":1}
I20260809 04:45:02.473471 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.467710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:46117 (local address 127.7.171.1:46497)
0809 04:45:02.468269 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:02.468286 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:02.468428 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:02.468775 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:02.468778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:02.468794 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:02.468938 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.468945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.470169 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.470172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.471728 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:02.471741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.472757 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.472762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:02.472994 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.473042 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 04:45:02.473145 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 04:45:02.473258 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":240,"thread_start_us":151,"threads_started":1}
I20260809 04:45:02.474699 8939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df32799d8f904b36bc92735a280d4fc5" candidate_uuid: "18b41b23cb4f41f29ac3132e40b7da49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d020a0c08f41b3bdfe2522eb4c595b" is_pre_election: true
I20260809 04:45:02.474699 8795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df32799d8f904b36bc92735a280d4fc5" candidate_uuid: "18b41b23cb4f41f29ac3132e40b7da49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f" is_pre_election: true
I20260809 04:45:02.475179 8939 raft_consensus.cc:2454] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18b41b23cb4f41f29ac3132e40b7da49 in term 0.
I20260809 04:45:02.475175 8795 raft_consensus.cc:2454] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18b41b23cb4f41f29ac3132e40b7da49 in term 0.
I20260809 04:45:02.475840 8581 leader_election.cc:302] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 18b41b23cb4f41f29ac3132e40b7da49, 24d020a0c08f41b3bdfe2522eb4c595b, e0cec089ba3341b1ba9e6f9e5b96c73f; no voters:
I20260809 04:45:02.476158 9014 raft_consensus.cc:2781] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:02.476269 9014 raft_consensus.cc:482] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:02.476337 9014 raft_consensus.cc:3037] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:02.477504 9014 raft_consensus.cc:504] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 46117 } } peers { permanent_uuid: "24d020a0c08f41b3bdfe2522eb4c595b" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36181 } } peers { permanent_uuid: "18b41b23cb4f41f29ac3132e40b7da49" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42879 } }
I20260809 04:45:02.477921 9014 leader_election.cc:288] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [CANDIDATE]: Term 1 election: Requested vote from peers e0cec089ba3341b1ba9e6f9e5b96c73f (127.7.171.2:46117), 24d020a0c08f41b3bdfe2522eb4c595b (127.7.171.3:36181)
I20260809 04:45:02.478339 8795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df32799d8f904b36bc92735a280d4fc5" candidate_uuid: "18b41b23cb4f41f29ac3132e40b7da49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f"
I20260809 04:45:02.478343 8939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df32799d8f904b36bc92735a280d4fc5" candidate_uuid: "18b41b23cb4f41f29ac3132e40b7da49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24d020a0c08f41b3bdfe2522eb4c595b"
I20260809 04:45:02.478489 8795 raft_consensus.cc:3037] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:02.478489 8939 raft_consensus.cc:3037] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:02.480015 8939 raft_consensus.cc:2454] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18b41b23cb4f41f29ac3132e40b7da49 in term 1.
I20260809 04:45:02.480193 8795 raft_consensus.cc:2454] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18b41b23cb4f41f29ac3132e40b7da49 in term 1.
I20260809 04:45:02.480509 8580 leader_election.cc:302] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18b41b23cb4f41f29ac3132e40b7da49, 24d020a0c08f41b3bdfe2522eb4c595b; no voters:
I20260809 04:45:02.480754 9014 raft_consensus.cc:2781] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:02.481112 9014 raft_consensus.cc:686] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [term 1 LEADER]: Becoming Leader. State: Replica: 18b41b23cb4f41f29ac3132e40b7da49, State: Running, Role: LEADER
I20260809 04:45:02.481482 9014 consensus_queue.cc:237] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 46117 } } peers { permanent_uuid: "24d020a0c08f41b3bdfe2522eb4c595b" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36181 } } peers { permanent_uuid: "18b41b23cb4f41f29ac3132e40b7da49" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42879 } }
I20260809 04:45:02.485035 8489 catalog_manager.cc:5495] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18b41b23cb4f41f29ac3132e40b7da49 (127.7.171.1). New cstate: current_term: 1 leader_uuid: "18b41b23cb4f41f29ac3132e40b7da49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0cec089ba3341b1ba9e6f9e5b96c73f" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 46117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24d020a0c08f41b3bdfe2522eb4c595b" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18b41b23cb4f41f29ac3132e40b7da49" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42879 } health_report { overall_health: HEALTHY } } }
W20260809 04:45:02.511363 8991 tablet.cc:2369] T df32799d8f904b36bc92735a280d4fc5 P 24d020a0c08f41b3bdfe2522eb4c595b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 04:45:02 dist-test-slave-1zqn krb5kdc[8441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:45:02.516952 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.507280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.7.171.1:42879)
0809 04:45:02.507461 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.507466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.507620 (+ 154us) server_negotiation.cc:407] Connection header received
0809 04:45:02.507809 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.507812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.507858 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.507946 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:02.508730 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.509320 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.510184 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.510419 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.510543 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.513286 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:02.513300 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:02.513316 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:02.513342 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:02.514995 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.515501 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.515507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.515508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.515596 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.515882 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.515886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.515887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.516231 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:02.516456 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.516639 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.516770 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 04:45:02.529630 8845 tablet.cc:2369] T df32799d8f904b36bc92735a280d4fc5 P e0cec089ba3341b1ba9e6f9e5b96c73f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 04:45:02 dist-test-slave-1zqn krb5kdc[8441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260809 04:45:02.543093 8699 tablet.cc:2369] T df32799d8f904b36bc92735a280d4fc5 P 18b41b23cb4f41f29ac3132e40b7da49: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 04:45:02 dist-test-slave-1zqn krb5kdc[8441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:45:02.548841 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.539666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44806 (local address 127.7.171.1:42879)
0809 04:45:02.539835 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.539841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.540027 (+ 186us) server_negotiation.cc:407] Connection header received
0809 04:45:02.540234 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.540238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.540289 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.540372 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:02.541101 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.541695 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.542566 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.542797 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.543020 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.545613 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:02.545616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:02.545619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:02.545645 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:02.547054 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.547486 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.547490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.547491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.547606 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.547851 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.547853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.547854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.548019 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:02.548230 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.548492 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.548639 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 04:45:02 dist-test-slave-1zqn krb5kdc[8441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:45:02 dist-test-slave-1zqn krb5kdc[8441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:45:02.578590 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.568988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44816 (local address 127.7.171.1:42879)
0809 04:45:02.569149 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.569154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.569335 (+ 181us) server_negotiation.cc:407] Connection header received
0809 04:45:02.569455 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.569457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.569502 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.569606 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:02.570282 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.570807 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.571600 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.571811 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.571887 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.574908 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:02.574914 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:02.574919 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:02.574974 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:02.576688 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.577247 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.577251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.577252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.577344 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.577616 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.577618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.577620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.577830 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:02.577968 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.578314 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.578458 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 04:45:02.580466 8606 tablet_service.cc:2908] Rejecting scan request for tablet df32799d8f904b36bc92735a280d4fc5: Uninitialized: clean time has not yet been initialized
I20260809 04:45:02.582923 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 8557
I20260809 04:45:02.591912 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 8703
I20260809 04:45:02.598824 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 8850
I20260809 04:45:02.607203 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 8455
2026-08-09T04:45:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2636 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:02 dist-test-slave-1zqn krb5kdc[9033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:02 dist-test-slave-1zqn krb5kdc[9033](info): set up 2 sockets
Aug 09 04:45:02 dist-test-slave-1zqn krb5kdc[9033](info): commencing operation
krb5kdc: starting...
W20260809 04:45:03.759654 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:03 dist-test-slave-1zqn krb5kdc[9033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:03.895272 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:42665
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:42665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:04.010088 9046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:04.010435 9046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:04.010520 9046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:04.015125 9046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:04.015257 9046 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:45:04.015295 9046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:04.015340 9046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:04.015374 9046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:04.015410 9046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:04.021045 9046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:42665
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:42665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:04.022215 9046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:04.023310 9046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:04.031982 9051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:04.032115 9052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:04.032001 9055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:04.032075 9046 server_base.cc:1034] running on GCE node
I20260809 04:45:04.032823 9046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:04.034180 9046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:04.035362 9046 hybrid_clock.cc:648] HybridClock initialized: now 1786250704035321 us; error 62 us; skew 500 ppm
Aug 09 04:45:04 dist-test-slave-1zqn krb5kdc[9033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:04.038329 9046 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:45:04.039184 9046 webserver.cc:460] Webserver started at http://127.7.171.62:38655/ using document root <none> and password file <none>
I20260809 04:45:04.039707 9046 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:04.039767 9046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:04.039930 9046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:04.042080 9046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "495a26e48d234fe5bd1e85e07136a375"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "1e7af47d6bbe05009b41363bb7aef3c0"
server_key_iv: "cc24d4e551c26ebf1cf3c83fae98fc5b"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.042686 9046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "495a26e48d234fe5bd1e85e07136a375"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "1e7af47d6bbe05009b41363bb7aef3c0"
server_key_iv: "cc24d4e551c26ebf1cf3c83fae98fc5b"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.046586 9046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260809 04:45:04.049065 9063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:04.050282 9046 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:04.050395 9046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "495a26e48d234fe5bd1e85e07136a375"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "1e7af47d6bbe05009b41363bb7aef3c0"
server_key_iv: "cc24d4e551c26ebf1cf3c83fae98fc5b"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.050482 9046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:04.067494 9046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:04.071082 9046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:04.071251 9046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:04.080654 9046 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:42665
I20260809 04:45:04.080642 9130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:42665 every 8 connection(s)
I20260809 04:45:04.081743 9046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:04.082784 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 9046
I20260809 04:45:04.082885 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:04.083176 7852 external_mini_cluster.cc:1428] Setting key 3450de5741942f2ab16b1c119d84d9ea
I20260809 04:45:04.085536 9131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:04.093117 9131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375: Bootstrap starting.
Aug 09 04:45:04 dist-test-slave-1zqn krb5kdc[9033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:04.096393 9131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:04.097939 9131 log.cc:824] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:04.099993 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.084780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38404 (local address 127.7.171.62:42665)
0809 04:45:04.085252 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.085273 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.085321 (+ 48us) server_negotiation.cc:407] Connection header received
0809 04:45:04.086268 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.086294 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.086775 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.087195 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.087904 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.089685 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.090545 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.091314 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.091506 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.094156 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.094163 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.094181 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.094216 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.096493 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.096974 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.096984 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.096994 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.097182 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.097459 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.097463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.097465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.098046 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.098265 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.098571 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.098958 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":115,"threads_started":1}
I20260809 04:45:04.100570 9131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375: No bootstrap required, opened a new log
I20260809 04:45:04.104274 9131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495a26e48d234fe5bd1e85e07136a375" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 42665 } }
I20260809 04:45:04.104525 9131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:04.104566 9131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 495a26e48d234fe5bd1e85e07136a375, State: Initialized, Role: FOLLOWER
I20260809 04:45:04.104952 9131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495a26e48d234fe5bd1e85e07136a375" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 42665 } }
I20260809 04:45:04.105083 9131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:04.105129 9131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:04.105206 9131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:04.106240 9131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495a26e48d234fe5bd1e85e07136a375" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 42665 } }
I20260809 04:45:04.106573 9131 leader_election.cc:302] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 495a26e48d234fe5bd1e85e07136a375; no voters:
I20260809 04:45:04.106953 9131 leader_election.cc:288] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:04.107086 9136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:04.107391 9136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [term 1 LEADER]: Becoming Leader. State: Replica: 495a26e48d234fe5bd1e85e07136a375, State: Running, Role: LEADER
I20260809 04:45:04.107746 9136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495a26e48d234fe5bd1e85e07136a375" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 42665 } }
I20260809 04:45:04.108474 9131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:04.109608 9137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "495a26e48d234fe5bd1e85e07136a375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495a26e48d234fe5bd1e85e07136a375" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 42665 } } }
I20260809 04:45:04.109752 9137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:04.109603 9138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 495a26e48d234fe5bd1e85e07136a375. Latest consensus state: current_term: 1 leader_uuid: "495a26e48d234fe5bd1e85e07136a375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "495a26e48d234fe5bd1e85e07136a375" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 42665 } } }
I20260809 04:45:04.109951 9138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:04.110195 9146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:04.113956 9146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:04.121177 9146 catalog_manager.cc:1347] Generated new cluster ID: 7fbf05bdc2e845998d850597221ad181
I20260809 04:45:04.121268 9146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:04.138370 9146 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:04.139770 9146 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:04.150023 9146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375: Generated new TSK 0
I20260809 04:45:04.150933 9146 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:45:04.192167 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:42665
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:04.312250 9159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:04.312546 9159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:04.312639 9159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:04.316695 9159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:04.316787 9159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:04.316859 9159 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:45:04.316941 9159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:04.322249 9159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.171.62:42665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:04.323354 9159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:04.324460 9159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:04.334216 9166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:04.334215 9168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:04.334218 9165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:04.334507 9159 server_base.cc:1034] running on GCE node
I20260809 04:45:04.334918 9159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:04.335498 9159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:04.336658 9159 hybrid_clock.cc:648] HybridClock initialized: now 1786250704336638 us; error 33 us; skew 500 ppm
Aug 09 04:45:04 dist-test-slave-1zqn krb5kdc[9033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:04.339536 9159 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:45:04.340417 9159 webserver.cc:460] Webserver started at http://127.7.171.1:35047/ using document root <none> and password file <none>
I20260809 04:45:04.341044 9159 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:04.341117 9159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:04.341281 9159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:04.343302 9159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "246ede9415ea4cfdad2429d36503439c"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "db602b0b3bb94037632415a2f29a123a"
server_key_iv: "81559b17dde64f1de330cec66c7211ce"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.343811 9159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "246ede9415ea4cfdad2429d36503439c"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "db602b0b3bb94037632415a2f29a123a"
server_key_iv: "81559b17dde64f1de330cec66c7211ce"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.347697 9159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 04:45:04.350103 9177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:04.351290 9159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:45:04.351423 9159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "246ede9415ea4cfdad2429d36503439c"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "db602b0b3bb94037632415a2f29a123a"
server_key_iv: "81559b17dde64f1de330cec66c7211ce"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.351536 9159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:04.373692 9159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:04.377238 9159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:04.377455 9159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:04.378945 9159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:04.378993 9159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:04.379033 9159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:04.379048 9159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:45:04 dist-test-slave-1zqn krb5kdc[9033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:04.390307 9159 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:41225
I20260809 04:45:04.390376 9304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:41225 every 8 connection(s)
I20260809 04:45:04.391506 9159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:04.394302 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.379943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:40719 (local address 127.7.171.62:42665)
0809 04:45:04.380131 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.380136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.381044 (+ 908us) server_negotiation.cc:407] Connection header received
0809 04:45:04.382235 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.382239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.382309 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.382423 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.383930 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.384836 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.386758 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.387112 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.387203 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.390603 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.390608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.390610 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.390640 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.392377 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.392908 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.392911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.392912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.392993 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.393294 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.393297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.393298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.393483 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.393560 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.394064 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.394179 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:45:04.395147 9187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.380397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:42665 (local address 127.7.171.1:40719)
0809 04:45:04.380854 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.380919 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.381949 (+ 1030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.382638 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.382647 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.383057 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:04.383774 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.383799 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.384994 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.385001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.386605 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.386623 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.387292 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.387297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.387526 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.388324 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:04.388352 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:04.390401 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:04.392517 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.392522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.392536 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.392821 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.393073 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.393076 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.393077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.393209 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.393604 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:04.393609 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:04.393839 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.394023 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.394289 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":133,"threads_started":1}
I20260809 04:45:04.396507 9305 heartbeater.cc:347] Connected to a master server at 127.7.171.62:42665
I20260809 04:45:04.396714 9305 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:04.397380 9305 heartbeater.cc:511] Master 127.7.171.62:42665 requested a full tablet report, sending...
I20260809 04:45:04.399150 9082 ts_manager.cc:194] Registered new tserver with Master: 246ede9415ea4cfdad2429d36503439c (127.7.171.1:41225)
I20260809 04:45:04.400297 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 9159
I20260809 04:45:04.400417 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:04.400663 9082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:40719
I20260809 04:45:04.400709 7852 external_mini_cluster.cc:1428] Setting key f14a012111936a1d490e3f88d8b03810
I20260809 04:45:04.407160 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.398853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:42665 (local address 127.7.171.1:37047)
0809 04:45:04.399212 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.399228 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.399329 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.399554 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.399557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.399771 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:04.399890 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.399898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.400958 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.400961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.401686 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.401696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.402284 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.402288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.402390 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.402942 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:04.402962 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:04.403446 (+ 484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:04.405557 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.405560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.405561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.405747 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.406055 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.406058 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.406059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.406135 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.406605 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:04.406608 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:04.406685 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.406777 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.406883 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":122,"threads_started":1}
I20260809 04:45:04.407330 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.398934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:37047 (local address 127.7.171.62:42665)
0809 04:45:04.399052 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.399055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.399368 (+ 313us) server_negotiation.cc:407] Connection header received
0809 04:45:04.399405 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.399407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.399444 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.399512 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.400023 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.400815 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.401808 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.402142 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.402264 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.403585 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.403588 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.403590 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.403611 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.405440 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.405845 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.405849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.405851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.405948 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.406230 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.406237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.406241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.406482 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.406592 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.406864 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.407088 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":41}
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:45:04.435314 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:42665
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:04.557098 9314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:04.557389 9314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:04.557440 9314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:04.562162 9314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:04.562245 9314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:04.562291 9314 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:45:04.562366 9314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:04.569160 9314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.171.62:42665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:04.570371 9314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:04.571498 9314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:04.581990 9322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:04.582007 9325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:04.582057 9323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:04.582269 9314 server_base.cc:1034] running on GCE node
I20260809 04:45:04.582767 9314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:04.583535 9314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:04.584733 9314 hybrid_clock.cc:648] HybridClock initialized: now 1786250704584717 us; error 40 us; skew 500 ppm
Aug 09 04:45:04 dist-test-slave-1zqn krb5kdc[9033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:04.587711 9314 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:45:04.588773 9314 webserver.cc:460] Webserver started at http://127.7.171.2:43999/ using document root <none> and password file <none>
I20260809 04:45:04.589408 9314 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:04.589509 9314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:04.589761 9314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:04.591905 9314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "5be24e26ce0349439f07738f49a606cc"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "858dfb8ac016f01b78a39a6ed48bfdfb"
server_key_iv: "2561d0f02ad19ecae925125be708a568"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.592572 9314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "5be24e26ce0349439f07738f49a606cc"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "858dfb8ac016f01b78a39a6ed48bfdfb"
server_key_iv: "2561d0f02ad19ecae925125be708a568"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.596524 9314 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:04.599380 9334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:04.600752 9314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:04.600894 9314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "5be24e26ce0349439f07738f49a606cc"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "858dfb8ac016f01b78a39a6ed48bfdfb"
server_key_iv: "2561d0f02ad19ecae925125be708a568"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.601011 9314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:04.619040 9314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:04.622509 9314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:04.622738 9314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:04.624399 9314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:04.624477 9314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:04.624550 9314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:04.624605 9314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:45:04 dist-test-slave-1zqn krb5kdc[9033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:04.636466 9314 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:35229
I20260809 04:45:04.637248 9460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:35229 every 8 connection(s)
I20260809 04:45:04.637606 9314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:04.639194 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.625382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:50205 (local address 127.7.171.62:42665)
0809 04:45:04.625564 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.625580 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.626504 (+ 924us) server_negotiation.cc:407] Connection header received
0809 04:45:04.627959 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.627963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.628031 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.628174 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.629870 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.630472 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.631686 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.632047 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.632196 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.634987 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.634995 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.634998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.635035 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.636830 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.637486 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.637489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.637491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.637581 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.638030 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.638033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.638034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.638264 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.638352 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.638941 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.639056 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:45:04.640023 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.625781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:42665 (local address 127.7.171.2:50205)
0809 04:45:04.626308 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.626372 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.627586 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.628382 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.628396 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.628916 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:04.629695 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.629723 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.630616 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.630620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.631546 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.631556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.632255 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.632259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.632462 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.633123 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:04.633148 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:04.634798 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:04.636954 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.636960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.636976 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.637359 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.637724 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.637727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.637729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.637914 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.638417 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:04.638423 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:04.638710 (+ 287us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.638873 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.639127 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":113,"threads_started":1}
I20260809 04:45:04.643252 9461 heartbeater.cc:347] Connected to a master server at 127.7.171.62:42665
I20260809 04:45:04.643410 9461 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:04.643447 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 9314
I20260809 04:45:04.643560 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:04.643883 7852 external_mini_cluster.cc:1428] Setting key afa7d1a0ea3cda315289b044fea1d7d1
I20260809 04:45:04.643949 9461 heartbeater.cc:511] Master 127.7.171.62:42665 requested a full tablet report, sending...
I20260809 04:45:04.645031 9082 ts_manager.cc:194] Registered new tserver with Master: 5be24e26ce0349439f07738f49a606cc (127.7.171.2:35229)
I20260809 04:45:04.645803 9082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:50205
I20260809 04:45:04.651979 9468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.643565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:42665 (local address 127.7.171.2:40341)
0809 04:45:04.643877 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.643892 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.643978 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.644460 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.644463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.644691 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:04.644814 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.644823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.645777 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.645780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.646630 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.646641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.647279 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.647282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.647369 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.647942 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:04.647956 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:04.648389 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:04.650284 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.650287 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.650289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.650493 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.650796 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.650798 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.650800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.650876 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.651454 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:04.651456 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:04.651529 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.651653 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.651830 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":123,"threads_started":1}
I20260809 04:45:04.652119 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.643546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:40341 (local address 127.7.171.62:42665)
0809 04:45:04.644218 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.644222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.644236 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:04.644276 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.644282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.644333 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.644446 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.644941 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.645651 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.646762 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.647144 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.647271 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.648514 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.648517 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.648519 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.648541 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.650178 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.650592 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.650598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.650603 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.650698 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.650984 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.650990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.650995 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.651347 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.651471 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.651724 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.651877 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586}
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:45:04.676052 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:42665
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:04.794060 9472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:04.794353 9472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:04.794445 9472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:04.798621 9472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:04.798731 9472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:04.798800 9472 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:45:04.798887 9472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:04.804464 9472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.171.62:42665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:04.805636 9472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:04.806737 9472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:04.817339 9478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:04.817358 9480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:04.817330 9477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:04.817693 9472 server_base.cc:1034] running on GCE node
I20260809 04:45:04.818233 9472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:04.818886 9472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:04.820164 9472 hybrid_clock.cc:648] HybridClock initialized: now 1786250704820126 us; error 49 us; skew 500 ppm
Aug 09 04:45:04 dist-test-slave-1zqn krb5kdc[9033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:04.823141 9472 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:45:04.824157 9472 webserver.cc:460] Webserver started at http://127.7.171.3:39045/ using document root <none> and password file <none>
I20260809 04:45:04.824764 9472 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:04.824828 9472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:04.824996 9472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:04.827092 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "a43799bddc9947ed9bda1167eb307871"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "0db2e809c27150ba2c43f6bae5f6fdb9"
server_key_iv: "18473dfe2fd9557214b7c0b2d425fbee"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.827616 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "a43799bddc9947ed9bda1167eb307871"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "0db2e809c27150ba2c43f6bae5f6fdb9"
server_key_iv: "18473dfe2fd9557214b7c0b2d425fbee"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.831611 9472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 04:45:04.834302 9488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:04.835601 9472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:45:04.835728 9472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "a43799bddc9947ed9bda1167eb307871"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1zqn"
server_key: "0db2e809c27150ba2c43f6bae5f6fdb9"
server_key_iv: "18473dfe2fd9557214b7c0b2d425fbee"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.835842 9472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:04.863634 9472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:04.866921 9472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:04.867130 9472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:04.868743 9472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:04.868798 9472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:04.868839 9472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:04.868876 9472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:45:04 dist-test-slave-1zqn krb5kdc[9033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:04.880338 9472 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:42811
I20260809 04:45:04.880414 9615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:42811 every 8 connection(s)
I20260809 04:45:04.881474 9472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:04.882910 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.869595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:39419 (local address 127.7.171.62:42665)
0809 04:45:04.869757 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.869762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.870658 (+ 896us) server_negotiation.cc:407] Connection header received
0809 04:45:04.871705 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.871709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.871777 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.871859 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.873363 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.874040 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.875491 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.875721 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.875865 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.878878 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.878885 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.878890 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.878926 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.880672 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.881163 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.881167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.881168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.881257 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.881605 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.881608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.881610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.881861 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.882077 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.882676 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.882787 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:45:04.883704 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.870026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:42665 (local address 127.7.171.3:39419)
0809 04:45:04.870483 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.870533 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.871409 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.872302 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.872309 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.872645 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:04.873217 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.873238 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.874182 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.874186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.875348 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.875359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.875924 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.875929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.876148 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.876814 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:04.876842 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:04.878689 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:04.880782 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.880787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.880798 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.881043 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.881352 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.881354 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.881356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.881482 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.882146 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:04.882155 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:04.882433 (+ 278us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.882595 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.882836 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":122,"threads_started":1}
I20260809 04:45:04.883972 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 9472
I20260809 04:45:04.884122 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:04.884472 7852 external_mini_cluster.cc:1428] Setting key 2798c223e85b7a900669dc90cfdcd793
I20260809 04:45:04.885232 9616 heartbeater.cc:347] Connected to a master server at 127.7.171.62:42665
I20260809 04:45:04.885385 9616 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:04.885941 9616 heartbeater.cc:511] Master 127.7.171.62:42665 requested a full tablet report, sending...
I20260809 04:45:04.886982 9082 ts_manager.cc:194] Registered new tserver with Master: a43799bddc9947ed9bda1167eb307871 (127.7.171.3:42811)
I20260809 04:45:04.887741 9082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:39419
I20260809 04:45:04.896652 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.887443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:42665 (local address 127.7.171.3:60797)
0809 04:45:04.887856 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.887876 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.888094 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.889749 (+ 1655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.889751 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.889969 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:04.890083 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.890089 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.890830 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.890833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.891543 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.891554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.891991 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.891994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.892144 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.892622 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:04.892633 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:04.893111 (+ 478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:04.895047 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.895051 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.895052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.895270 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.895572 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.895576 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.895578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.895687 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.896161 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:04.896163 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:04.896269 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.896365 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.896491 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":105,"threads_started":1}
I20260809 04:45:04.896694 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.887562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:60797 (local address 127.7.171.62:42665)
0809 04:45:04.887691 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.887695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.888040 (+ 345us) server_negotiation.cc:407] Connection header received
0809 04:45:04.888211 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.888214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.888263 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.888353 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.890203 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.890691 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.891660 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.891860 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.891937 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.893243 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.893246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.893248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.893270 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.894850 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.895367 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.895369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.895370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.895443 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.895794 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.895797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.895798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.895955 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.896162 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.896430 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.896534 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:45:04.899060 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:45:04 dist-test-slave-1zqn krb5kdc[9033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:45:04 dist-test-slave-1zqn krb5kdc[9033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:04.928862 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.920138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38412 (local address 127.7.171.62:42665)
0809 04:45:04.920273 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.920277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.920289 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:04.920333 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.920336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.920377 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.920473 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.921225 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.921770 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.922631 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.922852 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.922966 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.925510 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.925513 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.925515 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.925544 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.926963 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.927435 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.927439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.927440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.927519 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.927816 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.927820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.927822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.928020 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.928298 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.928558 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.928684 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:45:04.932255 9082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:04.935320 9082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:04.950208 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.945376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38088 (local address 127.7.171.2:35229)
0809 04:45:04.945804 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.945817 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.945841 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:45:04.945918 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.945924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.946105 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.946268 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:04.946455 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.947244 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.948898 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.949526 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.949651 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.949821 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.949960 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.950050 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":107,"threads_started":1}
I20260809 04:45:04.950197 9636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.944978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:35229 (local address 127.0.0.1:38088)
0809 04:45:04.945562 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.945576 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.945677 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.946248 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.946250 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.946269 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:04.946358 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.946365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.947365 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.947368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.948764 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.948775 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.949652 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.949655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.949758 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.949795 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.949917 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.950050 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":81,"thread_start_us":93,"threads_started":1}
I20260809 04:45:04.950489 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.944183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:42811 (local address 127.0.0.1:40490)
0809 04:45:04.944724 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.944780 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.944918 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.945784 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.945793 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.945818 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:04.945961 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.945972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.947418 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.947422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.948713 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.948726 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.949900 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.949904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.950011 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.950035 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.950105 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.950176 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260809 04:45:04.950788 9634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.944301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40490 (local address 127.7.171.3:42811)
0809 04:45:04.944665 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.944677 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.945073 (+ 396us) server_negotiation.cc:407] Connection header received
0809 04:45:04.945164 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.945173 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.945407 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.945598 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:04.946080 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.947279 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.948819 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.949554 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.949675 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.949832 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.950454 (+ 622us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.950650 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":137,"threads_started":1}
I20260809 04:45:04.950798 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.944996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36316 (local address 127.7.171.1:41225)
0809 04:45:04.945898 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.945904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.945933 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:45:04.946005 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.946009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.946166 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.946286 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:04.946675 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.947744 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.949129 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.949917 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.950014 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.950165 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.950571 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.950674 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":85,"threads_started":1}
I20260809 04:45:04.951112 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.944815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:41225 (local address 127.0.0.1:36316)
0809 04:45:04.945158 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.945171 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.945248 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.946420 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.946424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.946442 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:04.946545 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.946552 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.947868 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.947871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.948999 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.949008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.950227 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.950231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.950325 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.950358 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.950434 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.950496 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":4,"thread_start_us":98,"threads_started":1}
I20260809 04:45:04.953751 9387 tablet_service.cc:1467] Processing CreateTablet for tablet 7f901daea2ed4b90b1a321515d276dca (DEFAULT_TABLE table=test-table [id=0f50fa44584d4770baad7205f1f4edaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:04.954053 9542 tablet_service.cc:1467] Processing CreateTablet for tablet 7f901daea2ed4b90b1a321515d276dca (DEFAULT_TABLE table=test-table [id=0f50fa44584d4770baad7205f1f4edaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:04.954954 9387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f901daea2ed4b90b1a321515d276dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:04.955144 9542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f901daea2ed4b90b1a321515d276dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:04.955008 9231 tablet_service.cc:1467] Processing CreateTablet for tablet 7f901daea2ed4b90b1a321515d276dca (DEFAULT_TABLE table=test-table [id=0f50fa44584d4770baad7205f1f4edaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:04.955969 9231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f901daea2ed4b90b1a321515d276dca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:04.961611 9639 tablet_bootstrap.cc:492] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871: Bootstrap starting.
I20260809 04:45:04.963280 9641 tablet_bootstrap.cc:492] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc: Bootstrap starting.
I20260809 04:45:04.963872 9639 tablet_bootstrap.cc:654] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:04.964367 9640 tablet_bootstrap.cc:492] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c: Bootstrap starting.
I20260809 04:45:04.964658 9639 log.cc:824] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:04.965382 9641 tablet_bootstrap.cc:654] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:04.966203 9641 log.cc:824] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:04.966580 9640 tablet_bootstrap.cc:654] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:04.967146 9639 tablet_bootstrap.cc:492] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871: No bootstrap required, opened a new log
I20260809 04:45:04.967304 9639 ts_tablet_manager.cc:1397] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260809 04:45:04.967314 9640 log.cc:824] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:04.968509 9641 tablet_bootstrap.cc:492] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc: No bootstrap required, opened a new log
I20260809 04:45:04.968786 9641 ts_tablet_manager.cc:1397] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:45:04.969188 9640 tablet_bootstrap.cc:492] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c: No bootstrap required, opened a new log
I20260809 04:45:04.969365 9640 ts_tablet_manager.cc:1397] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:45:04.970268 9639 raft_consensus.cc:348] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } }
I20260809 04:45:04.970482 9639 raft_consensus.cc:374] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:04.970520 9639 raft_consensus.cc:729] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43799bddc9947ed9bda1167eb307871, State: Initialized, Role: FOLLOWER
I20260809 04:45:04.971024 9639 consensus_queue.cc:260] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } }
I20260809 04:45:04.971810 9616 heartbeater.cc:503] Master 127.7.171.62:42665 was elected leader, sending a full tablet report...
I20260809 04:45:04.972117 9639 ts_tablet_manager.cc:1428] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 04:45:04.973488 9641 raft_consensus.cc:348] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } }
I20260809 04:45:04.973834 9641 raft_consensus.cc:374] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:04.973933 9641 raft_consensus.cc:729] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5be24e26ce0349439f07738f49a606cc, State: Initialized, Role: FOLLOWER
I20260809 04:45:04.973829 9640 raft_consensus.cc:348] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } }
I20260809 04:45:04.974133 9640 raft_consensus.cc:374] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:04.974202 9640 raft_consensus.cc:729] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246ede9415ea4cfdad2429d36503439c, State: Initialized, Role: FOLLOWER
I20260809 04:45:04.974623 9641 consensus_queue.cc:260] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } }
I20260809 04:45:04.974606 9640 consensus_queue.cc:260] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } }
I20260809 04:45:04.975245 9305 heartbeater.cc:503] Master 127.7.171.62:42665 was elected leader, sending a full tablet report...
I20260809 04:45:04.975646 9641 ts_tablet_manager.cc:1428] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 04:45:04.976022 9461 heartbeater.cc:503] Master 127.7.171.62:42665 was elected leader, sending a full tablet report...
I20260809 04:45:04.976639 9640 ts_tablet_manager.cc:1428] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 04:45:05.021318 9646 raft_consensus.cc:482] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:05.021613 9646 raft_consensus.cc:504] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } }
I20260809 04:45:05.022818 9646 leader_election.cc:288] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a43799bddc9947ed9bda1167eb307871 (127.7.171.3:42811), 5be24e26ce0349439f07738f49a606cc (127.7.171.2:35229)
I20260809 04:45:05.027592 9634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.023263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:41095 (local address 127.7.171.3:42811)
0809 04:45:05.023443 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.023447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.023600 (+ 153us) server_negotiation.cc:407] Connection header received
0809 04:45:05.023816 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.023821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.023891 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.023989 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:05.024320 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.024883 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.026251 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.026877 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.026991 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.027059 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.027365 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.027460 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 04:45:05.027595 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.023134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:42811 (local address 127.7.171.1:41095)
0809 04:45:05.023500 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:05.023513 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:05.023661 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:05.024003 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:05.024006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:05.024026 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:05.024200 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.024206 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.024998 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.025004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.026126 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.026136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.027056 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.027059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.027187 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.027250 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 04:45:05.027337 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 04:45:05.027416 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":54,"thread_start_us":157,"threads_started":1}
I20260809 04:45:05.027637 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.023785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:43777 (local address 127.7.171.2:35229)
0809 04:45:05.023923 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.023927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.024123 (+ 196us) server_negotiation.cc:407] Connection header received
0809 04:45:05.024263 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.024266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.024318 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.024401 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:05.024670 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.025170 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.026480 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.027065 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.027178 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.027222 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.027461 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.027532 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:45:05.027638 9649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.023617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:35229 (local address 127.7.171.1:43777)
0809 04:45:05.024011 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:05.024024 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:05.024161 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:05.024415 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:05.024418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:05.024434 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:05.024563 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.024568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.025282 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.025286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.026303 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.026313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.027201 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.027204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.027349 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.027366 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 04:45:05.027432 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 04:45:05.027505 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":11,"thread_start_us":110,"threads_started":1}
I20260809 04:45:05.028386 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f901daea2ed4b90b1a321515d276dca" candidate_uuid: "246ede9415ea4cfdad2429d36503439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43799bddc9947ed9bda1167eb307871" is_pre_election: true
I20260809 04:45:05.028391 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f901daea2ed4b90b1a321515d276dca" candidate_uuid: "246ede9415ea4cfdad2429d36503439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5be24e26ce0349439f07738f49a606cc" is_pre_election: true
I20260809 04:45:05.028684 9566 raft_consensus.cc:2454] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246ede9415ea4cfdad2429d36503439c in term 0.
I20260809 04:45:05.028719 9409 raft_consensus.cc:2454] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246ede9415ea4cfdad2429d36503439c in term 0.
I20260809 04:45:05.029160 9180 leader_election.cc:302] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246ede9415ea4cfdad2429d36503439c, a43799bddc9947ed9bda1167eb307871; no voters:
I20260809 04:45:05.029472 9646 raft_consensus.cc:2781] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:05.029608 9646 raft_consensus.cc:482] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:05.029757 9646 raft_consensus.cc:3037] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:05.031093 9646 raft_consensus.cc:504] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } }
I20260809 04:45:05.031440 9646 leader_election.cc:288] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [CANDIDATE]: Term 1 election: Requested vote from peers a43799bddc9947ed9bda1167eb307871 (127.7.171.3:42811), 5be24e26ce0349439f07738f49a606cc (127.7.171.2:35229)
I20260809 04:45:05.031841 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f901daea2ed4b90b1a321515d276dca" candidate_uuid: "246ede9415ea4cfdad2429d36503439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43799bddc9947ed9bda1167eb307871"
I20260809 04:45:05.031894 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f901daea2ed4b90b1a321515d276dca" candidate_uuid: "246ede9415ea4cfdad2429d36503439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5be24e26ce0349439f07738f49a606cc"
I20260809 04:45:05.031965 9566 raft_consensus.cc:3037] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:05.032033 9409 raft_consensus.cc:3037] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:05.033566 9566 raft_consensus.cc:2454] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246ede9415ea4cfdad2429d36503439c in term 1.
I20260809 04:45:05.033582 9409 raft_consensus.cc:2454] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246ede9415ea4cfdad2429d36503439c in term 1.
I20260809 04:45:05.033900 9180 leader_election.cc:302] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246ede9415ea4cfdad2429d36503439c, a43799bddc9947ed9bda1167eb307871; no voters:
I20260809 04:45:05.034093 9646 raft_consensus.cc:2781] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:05.034334 9646 raft_consensus.cc:686] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 1 LEADER]: Becoming Leader. State: Replica: 246ede9415ea4cfdad2429d36503439c, State: Running, Role: LEADER
I20260809 04:45:05.034682 9646 consensus_queue.cc:237] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } }
I20260809 04:45:05.037940 9081 catalog_manager.cc:5495] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c reported cstate change: term changed from 0 to 1, leader changed from <none> to 246ede9415ea4cfdad2429d36503439c (127.7.171.1). New cstate: current_term: 1 leader_uuid: "246ede9415ea4cfdad2429d36503439c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } health_report { overall_health: HEALTHY } } }
I20260809 04:45:05.060806 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.056274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.7.171.1:41225)
0809 04:45:05.056467 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.056475 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.056624 (+ 149us) server_negotiation.cc:407] Connection header received
0809 04:45:05.056899 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.056902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.057037 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.057164 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:05.057495 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.058067 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.059066 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.059313 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.059437 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.059825 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:05.060322 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:05.060462 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.060565 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.060630 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 04:45:05.067719 9409 raft_consensus.cc:1261] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc [term 1 FOLLOWER]: Refusing update from remote peer 246ede9415ea4cfdad2429d36503439c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:05.067850 9566 raft_consensus.cc:1261] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871 [term 1 FOLLOWER]: Refusing update from remote peer 246ede9415ea4cfdad2429d36503439c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:05.068578 9646 consensus_queue.cc:1035] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:05.068823 9650 consensus_queue.cc:1035] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:05.088910 9080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:05.089114 9080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:05.091502 9080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375: Sending DeleteTablet for 3 replicas of tablet 7f901daea2ed4b90b1a321515d276dca
I20260809 04:45:05.092386 9387 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f901daea2ed4b90b1a321515d276dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:05 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:38088
I20260809 04:45:05.092782 9542 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f901daea2ed4b90b1a321515d276dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:05 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:40490
I20260809 04:45:05.092952 9231 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f901daea2ed4b90b1a321515d276dca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:05 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:36316
I20260809 04:45:05.093864 9665 tablet_replica.cc:330] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871: stopping tablet replica
I20260809 04:45:05.094048 9080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 04:45:05.094453 9665 raft_consensus.cc:2229] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 04:45:05.094502 9080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:05.094924 9665 raft_consensus.cc:2258] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:05.095391 9664 tablet_replica.cc:330] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc: stopping tablet replica
I20260809 04:45:05.095685 9666 tablet_replica.cc:330] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c: stopping tablet replica
I20260809 04:45:05.096325 9666 raft_consensus.cc:2229] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:05.096041 9664 raft_consensus.cc:2229] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:05.096828 9665 ts_tablet_manager.cc:1905] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:05.097294 9664 raft_consensus.cc:2258] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:05.097477 9666 raft_consensus.cc:2258] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:05.100281 9665 ts_tablet_manager.cc:1918] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:05.100389 9665 log.cc:1196] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/7f901daea2ed4b90b1a321515d276dca
I20260809 04:45:05.100606 9387 tablet_service.cc:1467] Processing CreateTablet for tablet 666974058ef248b9af78bbeda6cb0511 (DEFAULT_TABLE table=test-table [id=5f6ed6c6e86f4b23a396f7a89c532218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:05.100651 9542 tablet_service.cc:1467] Processing CreateTablet for tablet 666974058ef248b9af78bbeda6cb0511 (DEFAULT_TABLE table=test-table [id=5f6ed6c6e86f4b23a396f7a89c532218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:05.100899 9387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666974058ef248b9af78bbeda6cb0511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:05.100921 9542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666974058ef248b9af78bbeda6cb0511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:05.101789 9666 ts_tablet_manager.cc:1905] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:05.102473 9664 ts_tablet_manager.cc:1905] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:05.103333 9639 tablet_bootstrap.cc:492] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871: Bootstrap starting.
I20260809 04:45:05.104206 9231 tablet_service.cc:1467] Processing CreateTablet for tablet 666974058ef248b9af78bbeda6cb0511 (DEFAULT_TABLE table=test-table [id=5f6ed6c6e86f4b23a396f7a89c532218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:05.104401 9639 tablet_bootstrap.cc:654] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:05.104558 9231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666974058ef248b9af78bbeda6cb0511. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:05.104842 9665 ts_tablet_manager.cc:1939] T 7f901daea2ed4b90b1a321515d276dca P a43799bddc9947ed9bda1167eb307871: Deleting consensus metadata
I20260809 04:45:05.105432 9664 ts_tablet_manager.cc:1918] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:05.105651 9664 log.cc:1196] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/7f901daea2ed4b90b1a321515d276dca
I20260809 04:45:05.106101 9664 ts_tablet_manager.cc:1939] T 7f901daea2ed4b90b1a321515d276dca P 5be24e26ce0349439f07738f49a606cc: Deleting consensus metadata
I20260809 04:45:05.106156 9067 catalog_manager.cc:4841] TS a43799bddc9947ed9bda1167eb307871 (127.7.171.3:42811): tablet 7f901daea2ed4b90b1a321515d276dca (table test-table [id=0f50fa44584d4770baad7205f1f4edaf]) successfully deleted
I20260809 04:45:05.106432 9666 ts_tablet_manager.cc:1918] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:05.106554 9666 log.cc:1196] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/wal/wals/7f901daea2ed4b90b1a321515d276dca
I20260809 04:45:05.106863 9666 ts_tablet_manager.cc:1939] T 7f901daea2ed4b90b1a321515d276dca P 246ede9415ea4cfdad2429d36503439c: Deleting consensus metadata
I20260809 04:45:05.107686 9067 catalog_manager.cc:4841] TS 5be24e26ce0349439f07738f49a606cc (127.7.171.2:35229): tablet 7f901daea2ed4b90b1a321515d276dca (table test-table [id=0f50fa44584d4770baad7205f1f4edaf]) successfully deleted
I20260809 04:45:05.107792 9640 tablet_bootstrap.cc:492] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c: Bootstrap starting.
I20260809 04:45:05.108202 9641 tablet_bootstrap.cc:492] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc: Bootstrap starting.
I20260809 04:45:05.108888 9640 tablet_bootstrap.cc:654] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:05.109032 9639 tablet_bootstrap.cc:492] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871: No bootstrap required, opened a new log
I20260809 04:45:05.109001 9068 catalog_manager.cc:4841] TS 246ede9415ea4cfdad2429d36503439c (127.7.171.1:41225): tablet 7f901daea2ed4b90b1a321515d276dca (table test-table [id=0f50fa44584d4770baad7205f1f4edaf]) successfully deleted
I20260809 04:45:05.109115 9639 ts_tablet_manager.cc:1397] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260809 04:45:05.109272 9641 tablet_bootstrap.cc:654] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:05.109678 9639 raft_consensus.cc:348] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.109803 9639 raft_consensus.cc:374] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:05.109849 9639 raft_consensus.cc:729] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43799bddc9947ed9bda1167eb307871, State: Initialized, Role: FOLLOWER
I20260809 04:45:05.110004 9639 consensus_queue.cc:260] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.110262 9639 ts_tablet_manager.cc:1428] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 04:45:05.110679 9640 tablet_bootstrap.cc:492] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c: No bootstrap required, opened a new log
I20260809 04:45:05.110756 9640 ts_tablet_manager.cc:1397] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:05.111115 9641 tablet_bootstrap.cc:492] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc: No bootstrap required, opened a new log
I20260809 04:45:05.111202 9641 ts_tablet_manager.cc:1397] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:05.111266 9640 raft_consensus.cc:348] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.111372 9640 raft_consensus.cc:374] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:05.111402 9640 raft_consensus.cc:729] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246ede9415ea4cfdad2429d36503439c, State: Initialized, Role: FOLLOWER
I20260809 04:45:05.111541 9640 consensus_queue.cc:260] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.111656 9641 raft_consensus.cc:348] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.111759 9641 raft_consensus.cc:374] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:05.111789 9641 raft_consensus.cc:729] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5be24e26ce0349439f07738f49a606cc, State: Initialized, Role: FOLLOWER
I20260809 04:45:05.111901 9641 consensus_queue.cc:260] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.112185 9641 ts_tablet_manager.cc:1428] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:05.112607 9640 ts_tablet_manager.cc:1428] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260809 04:45:05.132557 9617 tablet.cc:2369] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:45:05.138893 9462 tablet.cc:2369] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:45:05.143007 9306 tablet.cc:2369] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:45:05.256196 9647 raft_consensus.cc:482] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:05.256362 9647 raft_consensus.cc:504] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.257512 9647 leader_election.cc:288] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 246ede9415ea4cfdad2429d36503439c (127.7.171.1:41225), a43799bddc9947ed9bda1167eb307871 (127.7.171.3:42811)
I20260809 04:45:05.261546 9634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.258005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:38135 (local address 127.7.171.3:42811)
0809 04:45:05.258150 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.258155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.258311 (+ 156us) server_negotiation.cc:407] Connection header received
0809 04:45:05.258501 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.258504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.258555 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.258653 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:05.258933 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.259429 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.260601 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.261004 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.261119 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.261180 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.261355 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.261421 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:45:05.261606 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.257829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:42811 (local address 127.7.171.2:38135)
0809 04:45:05.258217 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:05.258228 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:05.258384 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:05.258668 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:05.258671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:05.258693 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:05.258821 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.258827 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.259544 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.259548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.260485 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.260493 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.261126 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.261130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.261228 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.261254 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:45:05.261338 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:45:05.261404 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":59,"spinlock_wait_cycles":7680,"thread_start_us":83,"threads_started":1}
I20260809 04:45:05.261727 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.258006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:50911 (local address 127.7.171.1:41225)
0809 04:45:05.258138 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.258144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.258298 (+ 154us) server_negotiation.cc:407] Connection header received
0809 04:45:05.258499 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.258502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.258558 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.258656 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:05.258936 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.259434 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.260642 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.261155 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.261269 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.261330 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.261528 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.261604 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:45:05.262094 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.257829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:41225 (local address 127.7.171.2:50911)
0809 04:45:05.258203 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:05.258216 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:05.258383 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:05.258669 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:05.258672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:05.258697 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:05.258824 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.258829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.259553 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.259557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.260505 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.260515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.261275 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.261278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.261378 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.261401 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:45:05.261861 (+ 460us) client_negotiation.cc:240] Negotiation successful
0809 04:45:05.261931 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":57,"spinlock_wait_cycles":26112,"thread_start_us":77,"threads_started":1}
I20260809 04:45:05.262166 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666974058ef248b9af78bbeda6cb0511" candidate_uuid: "5be24e26ce0349439f07738f49a606cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43799bddc9947ed9bda1167eb307871" is_pre_election: true
I20260809 04:45:05.262315 9566 raft_consensus.cc:2454] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5be24e26ce0349439f07738f49a606cc in term 0.
I20260809 04:45:05.262641 9254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666974058ef248b9af78bbeda6cb0511" candidate_uuid: "5be24e26ce0349439f07738f49a606cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246ede9415ea4cfdad2429d36503439c" is_pre_election: true
I20260809 04:45:05.262725 9337 leader_election.cc:302] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5be24e26ce0349439f07738f49a606cc, a43799bddc9947ed9bda1167eb307871; no voters:
I20260809 04:45:05.262890 9254 raft_consensus.cc:2454] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5be24e26ce0349439f07738f49a606cc in term 0.
I20260809 04:45:05.263011 9647 raft_consensus.cc:2781] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:05.263070 9647 raft_consensus.cc:482] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:05.263120 9647 raft_consensus.cc:3037] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:05.264216 9647 raft_consensus.cc:504] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.264565 9647 leader_election.cc:288] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [CANDIDATE]: Term 1 election: Requested vote from peers 246ede9415ea4cfdad2429d36503439c (127.7.171.1:41225), a43799bddc9947ed9bda1167eb307871 (127.7.171.3:42811)
I20260809 04:45:05.264878 9254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666974058ef248b9af78bbeda6cb0511" candidate_uuid: "5be24e26ce0349439f07738f49a606cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246ede9415ea4cfdad2429d36503439c"
I20260809 04:45:05.265024 9254 raft_consensus.cc:3037] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:05.265040 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666974058ef248b9af78bbeda6cb0511" candidate_uuid: "5be24e26ce0349439f07738f49a606cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43799bddc9947ed9bda1167eb307871"
I20260809 04:45:05.265161 9566 raft_consensus.cc:3037] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:05.266256 9566 raft_consensus.cc:2454] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5be24e26ce0349439f07738f49a606cc in term 1.
I20260809 04:45:05.266256 9254 raft_consensus.cc:2454] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5be24e26ce0349439f07738f49a606cc in term 1.
I20260809 04:45:05.266553 9338 leader_election.cc:302] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246ede9415ea4cfdad2429d36503439c, 5be24e26ce0349439f07738f49a606cc; no voters:
I20260809 04:45:05.266723 9647 raft_consensus.cc:2781] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:05.266937 9647 raft_consensus.cc:686] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 1 LEADER]: Becoming Leader. State: Replica: 5be24e26ce0349439f07738f49a606cc, State: Running, Role: LEADER
I20260809 04:45:05.267192 9647 consensus_queue.cc:237] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.269611 9081 catalog_manager.cc:5495] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5be24e26ce0349439f07738f49a606cc (127.7.171.2). New cstate: current_term: 1 leader_uuid: "5be24e26ce0349439f07738f49a606cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } health_report { overall_health: UNKNOWN } } }
Aug 09 04:45:05 dist-test-slave-1zqn krb5kdc[9033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:05.326925 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.317982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:42665 (local address 127.0.0.1:38424)
0809 04:45:05.318323 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:05.318337 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:05.318477 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:05.318783 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:05.318786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:05.319233 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:05.319327 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.319334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.320176 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.320182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.320749 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.320756 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.321196 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.321199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.321274 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.321855 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:05.321868 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:05.323375 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:05.325231 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:05.325244 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:05.325251 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:05.325489 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:05.325778 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:05.325781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:05.325782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:05.325864 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:05.326246 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:05.326256 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:05.326375 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 04:45:05.326490 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 04:45:05.326713 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":1,"thread_start_us":140,"threads_started":1}
I20260809 04:45:05.326925 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.318006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38424 (local address 127.7.171.62:42665)
0809 04:45:05.318155 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.318178 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.318423 (+ 245us) server_negotiation.cc:407] Connection header received
0809 04:45:05.318625 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.318628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.318675 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.318760 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:05.319455 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.319991 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.320866 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.321087 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.321190 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.323518 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:05.323523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:05.323524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:05.323544 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:05.325122 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:05.325599 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:05.325603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:05.325605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:05.325682 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:05.325948 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:05.325953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:05.325955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:05.326139 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:05.326245 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.326548 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.326766 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":45}
I20260809 04:45:05.331195 9081 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:38424:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:45:05.332172 9081 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:05.337296 9231 tablet_service.cc:1467] Processing CreateTablet for tablet db5f8a909f5a44a78d31eb73edf15f8f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a90392668bcd454fad1cd180de90acd5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:45:05.337405 9387 tablet_service.cc:1467] Processing CreateTablet for tablet db5f8a909f5a44a78d31eb73edf15f8f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a90392668bcd454fad1cd180de90acd5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:45:05.337406 9542 tablet_service.cc:1467] Processing CreateTablet for tablet db5f8a909f5a44a78d31eb73edf15f8f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a90392668bcd454fad1cd180de90acd5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:45:05.337766 9231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5f8a909f5a44a78d31eb73edf15f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:05.337785 9387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5f8a909f5a44a78d31eb73edf15f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:05.337785 9542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5f8a909f5a44a78d31eb73edf15f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:05.340667 9640 tablet_bootstrap.cc:492] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: Bootstrap starting.
I20260809 04:45:05.340803 9639 tablet_bootstrap.cc:492] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871: Bootstrap starting.
I20260809 04:45:05.341024 9641 tablet_bootstrap.cc:492] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc: Bootstrap starting.
I20260809 04:45:05.341652 9640 tablet_bootstrap.cc:654] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:05.341794 9639 tablet_bootstrap.cc:654] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:05.341907 9641 tablet_bootstrap.cc:654] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:05.343025 9640 tablet_bootstrap.cc:492] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: No bootstrap required, opened a new log
I20260809 04:45:05.343106 9640 ts_tablet_manager.cc:1397] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:05.343357 9639 tablet_bootstrap.cc:492] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871: No bootstrap required, opened a new log
I20260809 04:45:05.343457 9639 ts_tablet_manager.cc:1397] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:05.343521 9640 raft_consensus.cc:348] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.343662 9640 raft_consensus.cc:374] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:05.343705 9640 raft_consensus.cc:729] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246ede9415ea4cfdad2429d36503439c, State: Initialized, Role: FOLLOWER
I20260809 04:45:05.343834 9640 consensus_queue.cc:260] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.343943 9639 raft_consensus.cc:348] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.344092 9639 raft_consensus.cc:374] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:05.344129 9639 raft_consensus.cc:729] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43799bddc9947ed9bda1167eb307871, State: Initialized, Role: FOLLOWER
I20260809 04:45:05.344149 9640 ts_tablet_manager.cc:1428] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:05.344151 9650 tablet_replica.cc:439] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } }
I20260809 04:45:05.344241 9639 consensus_queue.cc:260] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.344318 9650 tablet_replica.cc:442] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:05.344508 9639 ts_tablet_manager.cc:1428] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 04:45:05.344745 9645 tablet_replica.cc:439] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } }
I20260809 04:45:05.344911 9645 tablet_replica.cc:442] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:05.345943 9641 tablet_bootstrap.cc:492] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc: No bootstrap required, opened a new log
I20260809 04:45:05.346127 9641 ts_tablet_manager.cc:1397] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 04:45:05.346652 9641 raft_consensus.cc:348] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.346796 9641 raft_consensus.cc:374] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:05.346832 9641 raft_consensus.cc:729] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5be24e26ce0349439f07738f49a606cc, State: Initialized, Role: FOLLOWER
I20260809 04:45:05.346976 9641 consensus_queue.cc:260] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.347272 9641 ts_tablet_manager.cc:1428] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 04:45:05.347270 9647 tablet_replica.cc:439] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } }
I20260809 04:45:05.347419 9647 tablet_replica.cc:442] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:05.545780 9650 raft_consensus.cc:482] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:05.545970 9650 raft_consensus.cc:504] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.546499 9650 leader_election.cc:288] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5be24e26ce0349439f07738f49a606cc (127.7.171.2:35229), a43799bddc9947ed9bda1167eb307871 (127.7.171.3:42811)
I20260809 04:45:05.547015 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5f8a909f5a44a78d31eb73edf15f8f" candidate_uuid: "246ede9415ea4cfdad2429d36503439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5be24e26ce0349439f07738f49a606cc" is_pre_election: true
I20260809 04:45:05.547109 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5f8a909f5a44a78d31eb73edf15f8f" candidate_uuid: "246ede9415ea4cfdad2429d36503439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43799bddc9947ed9bda1167eb307871" is_pre_election: true
I20260809 04:45:05.547191 9409 raft_consensus.cc:2454] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246ede9415ea4cfdad2429d36503439c in term 0.
I20260809 04:45:05.547302 9566 raft_consensus.cc:2454] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246ede9415ea4cfdad2429d36503439c in term 0.
I20260809 04:45:05.547549 9180 leader_election.cc:302] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246ede9415ea4cfdad2429d36503439c, 5be24e26ce0349439f07738f49a606cc; no voters:
I20260809 04:45:05.547744 9650 raft_consensus.cc:2781] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:05.547803 9650 raft_consensus.cc:482] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:05.547830 9650 raft_consensus.cc:3037] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:05.548971 9650 raft_consensus.cc:504] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.549296 9650 leader_election.cc:288] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [CANDIDATE]: Term 1 election: Requested vote from peers 5be24e26ce0349439f07738f49a606cc (127.7.171.2:35229), a43799bddc9947ed9bda1167eb307871 (127.7.171.3:42811)
I20260809 04:45:05.549649 9409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5f8a909f5a44a78d31eb73edf15f8f" candidate_uuid: "246ede9415ea4cfdad2429d36503439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5be24e26ce0349439f07738f49a606cc"
I20260809 04:45:05.549722 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5f8a909f5a44a78d31eb73edf15f8f" candidate_uuid: "246ede9415ea4cfdad2429d36503439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43799bddc9947ed9bda1167eb307871"
I20260809 04:45:05.549774 9409 raft_consensus.cc:3037] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:05.549844 9566 raft_consensus.cc:3037] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:05.550904 9566 raft_consensus.cc:2454] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246ede9415ea4cfdad2429d36503439c in term 1.
I20260809 04:45:05.550904 9409 raft_consensus.cc:2454] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246ede9415ea4cfdad2429d36503439c in term 1.
I20260809 04:45:05.551175 9180 leader_election.cc:302] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246ede9415ea4cfdad2429d36503439c, a43799bddc9947ed9bda1167eb307871; no voters:
I20260809 04:45:05.551326 9650 raft_consensus.cc:2781] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:05.551415 9650 raft_consensus.cc:686] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [term 1 LEADER]: Becoming Leader. State: Replica: 246ede9415ea4cfdad2429d36503439c, State: Running, Role: LEADER
I20260809 04:45:05.551556 9650 consensus_queue.cc:237] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } }
I20260809 04:45:05.552199 9650 tablet_replica.cc:439] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: TxnStatusTablet state changed. Reason: New leader 246ede9415ea4cfdad2429d36503439c. Latest consensus state: current_term: 1 leader_uuid: "246ede9415ea4cfdad2429d36503439c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } }
I20260809 04:45:05.552296 9650 tablet_replica.cc:442] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:45:05.552754 9687 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:45:05.552963 9687 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:45:05.553452 9081 catalog_manager.cc:5495] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c reported cstate change: term changed from 0 to 1, leader changed from <none> to 246ede9415ea4cfdad2429d36503439c (127.7.171.1). New cstate: current_term: 1 leader_uuid: "246ede9415ea4cfdad2429d36503439c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } health_report { overall_health: UNKNOWN } } }
I20260809 04:45:05.582809 9682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.578610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:41225 (local address 127.0.0.1:36328)
0809 04:45:05.578769 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:05.578784 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:05.578889 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:05.579319 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:05.579322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:05.579333 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:05.579448 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.579455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.580573 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.580576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.581321 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.581332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.581999 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.582003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.582096 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.582149 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:05.582520 (+ 371us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:05.582538 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 04:45:05.582613 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 04:45:05.582691 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 04:45:05.582839 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.578798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36328 (local address 127.7.171.1:41225)
0809 04:45:05.578946 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.578952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.579077 (+ 125us) server_negotiation.cc:407] Connection header received
0809 04:45:05.579135 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.579138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.579189 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.579257 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:05.579610 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.580458 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.581444 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.581798 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.581908 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.582303 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:05.582392 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:05.582468 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.582640 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.582702 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:45:05.586264 9409 raft_consensus.cc:1261] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc [term 1 FOLLOWER]: Refusing update from remote peer 246ede9415ea4cfdad2429d36503439c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:05.586321 9566 raft_consensus.cc:1261] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871 [term 1 FOLLOWER]: Refusing update from remote peer 246ede9415ea4cfdad2429d36503439c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:05.586771 9646 consensus_queue.cc:1035] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:05.586941 9650 consensus_queue.cc:1035] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:05.588662 9645 tablet_replica.cc:439] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871: TxnStatusTablet state changed. Reason: New leader 246ede9415ea4cfdad2429d36503439c. Latest consensus state: current_term: 1 leader_uuid: "246ede9415ea4cfdad2429d36503439c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } }
I20260809 04:45:05.588783 9645 tablet_replica.cc:442] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:05.589671 9646 tablet_replica.cc:439] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "246ede9415ea4cfdad2429d36503439c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } }
I20260809 04:45:05.589779 9646 tablet_replica.cc:442] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:45:05.589960 9647 tablet_replica.cc:439] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc: TxnStatusTablet state changed. Reason: New leader 246ede9415ea4cfdad2429d36503439c. Latest consensus state: current_term: 1 leader_uuid: "246ede9415ea4cfdad2429d36503439c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } }
I20260809 04:45:05.590080 9647 tablet_replica.cc:442] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:05.591248 9691 mvcc.cc:205] Tried to move back new op lower bound from 7316482890076041216 to 7316482889940303872. Current Snapshot: MvccSnapshot[applied={T|T < 7316482890076041216 or (T in {7316482890076041216})}]
I20260809 04:45:05.591424 9692 mvcc.cc:205] Tried to move back new op lower bound from 7316482890076041216 to 7316482889940303872. Current Snapshot: MvccSnapshot[applied={T|T < 7316482890076041216}]
I20260809 04:45:05.591952 9645 tablet_replica.cc:439] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "246ede9415ea4cfdad2429d36503439c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } }
I20260809 04:45:05.592087 9645 tablet_replica.cc:442] T db5f8a909f5a44a78d31eb73edf15f8f P a43799bddc9947ed9bda1167eb307871: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:05.596179 9650 tablet_replica.cc:439] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "246ede9415ea4cfdad2429d36503439c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } }
I20260809 04:45:05.596305 9650 tablet_replica.cc:442] T db5f8a909f5a44a78d31eb73edf15f8f P 246ede9415ea4cfdad2429d36503439c: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:45:05.597599 9695 mvcc.cc:205] Tried to move back new op lower bound from 7316482890076041216 to 7316482889940303872. Current Snapshot: MvccSnapshot[applied={T|T < 7316482890076041216}]
I20260809 04:45:05.597790 9647 tablet_replica.cc:439] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "246ede9415ea4cfdad2429d36503439c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 } } peers { permanent_uuid: "5be24e26ce0349439f07738f49a606cc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35229 } } peers { permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 } } }
I20260809 04:45:05.597922 9647 tablet_replica.cc:442] T db5f8a909f5a44a78d31eb73edf15f8f P 5be24e26ce0349439f07738f49a606cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:05.601966 9703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.597404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38092 (local address 127.7.171.2:35229)
0809 04:45:05.597663 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.597672 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.597687 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:05.597762 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.597765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.597837 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.598310 (+ 473us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:05.598416 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.599230 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.600445 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.600674 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.600754 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.601031 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:05.601507 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:05.601598 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.601691 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.601780 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":92,"threads_started":1}
I20260809 04:45:05.612584 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.609419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:35935 (local address 127.7.171.1:41225)
0809 04:45:05.609541 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.609545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.609558 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:45:05.609710 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.609718 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.609763 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.609826 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:05.610114 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.610599 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.611458 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.611677 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.611785 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.612115 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:05.612187 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:05.612265 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.612380 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.612453 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:45:05.612617 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.609149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:41225 (local address 127.7.171.2:35935)
0809 04:45:05.609456 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:05.609473 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:05.609573 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:05.609842 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:05.609845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:05.609854 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:05.609982 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.609988 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.610705 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.610710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.611341 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.611349 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.611794 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.611797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.611888 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.611930 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:05.612269 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:05.612278 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:45:05.612356 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 04:45:05.612443 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":100,"threads_started":1}
I20260809 04:45:05.619714 9566 raft_consensus.cc:1261] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871 [term 1 FOLLOWER]: Refusing update from remote peer 5be24e26ce0349439f07738f49a606cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:05.619992 9254 raft_consensus.cc:1261] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c [term 1 FOLLOWER]: Refusing update from remote peer 5be24e26ce0349439f07738f49a606cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:05.620260 9647 consensus_queue.cc:1035] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a43799bddc9947ed9bda1167eb307871" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:05.620520 9673 consensus_queue.cc:1035] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "246ede9415ea4cfdad2429d36503439c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:05.648474 9703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.641170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:40035 (local address 127.7.171.2:35229)
0809 04:45:05.641341 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.641346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.641614 (+ 268us) server_negotiation.cc:407] Connection header received
0809 04:45:05.641806 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.641809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.641865 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.641945 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:05.642287 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.644530 (+ 2243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.647134 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.647454 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.647546 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.647985 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:05.648090 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:05.648137 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.648289 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.648344 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 04:45:05.649149 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.640764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:35229 (local address 127.7.171.1:40035)
0809 04:45:05.641533 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:05.641549 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:05.641688 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:05.642060 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:05.642063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:05.642072 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:05.642175 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.642181 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.644683 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.644687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.647004 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:05.647015 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.647677 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.647681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:05.647777 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.647836 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:05.648199 (+ 363us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:05.648205 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:45:05.648898 (+ 693us) client_negotiation.cc:240] Negotiation successful
0809 04:45:05.648991 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":113,"threads_started":1}
I20260809 04:45:05.682116 9081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:05.682314 9081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:05.684806 9081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 495a26e48d234fe5bd1e85e07136a375: Sending DeleteTablet for 3 replicas of tablet 666974058ef248b9af78bbeda6cb0511
I20260809 04:45:05.685417 9231 tablet_service.cc:1514] Processing DeleteTablet for tablet 666974058ef248b9af78bbeda6cb0511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:05 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:36316
I20260809 04:45:05.685655 9542 tablet_service.cc:1514] Processing DeleteTablet for tablet 666974058ef248b9af78bbeda6cb0511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:05 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:40490
I20260809 04:45:05.685922 9387 tablet_service.cc:1514] Processing DeleteTablet for tablet 666974058ef248b9af78bbeda6cb0511 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:05 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:38088
I20260809 04:45:05.686329 9718 tablet_replica.cc:330] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c: stopping tablet replica
I20260809 04:45:05.686484 9718 raft_consensus.cc:2229] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:05.686602 9718 raft_consensus.cc:2258] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:05.687322 9718 ts_tablet_manager.cc:1905] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:05.687472 9720 tablet_replica.cc:330] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc: stopping tablet replica
I20260809 04:45:05.687654 9720 raft_consensus.cc:2229] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:05.688045 9720 raft_consensus.cc:2258] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:05.688834 9720 ts_tablet_manager.cc:1905] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:05.690559 9718 ts_tablet_manager.cc:1918] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:05.690644 9718 log.cc:1196] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-0/wal/wals/666974058ef248b9af78bbeda6cb0511
I20260809 04:45:05.690979 9718 ts_tablet_manager.cc:1939] T 666974058ef248b9af78bbeda6cb0511 P 246ede9415ea4cfdad2429d36503439c: Deleting consensus metadata
I20260809 04:45:05.691761 9068 catalog_manager.cc:4841] TS 246ede9415ea4cfdad2429d36503439c (127.7.171.1:41225): tablet 666974058ef248b9af78bbeda6cb0511 (table test-table [id=5f6ed6c6e86f4b23a396f7a89c532218]) successfully deleted
I20260809 04:45:05.692226 9720 ts_tablet_manager.cc:1918] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:05.692377 9720 log.cc:1196] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/666974058ef248b9af78bbeda6cb0511
I20260809 04:45:05.692764 9720 ts_tablet_manager.cc:1939] T 666974058ef248b9af78bbeda6cb0511 P 5be24e26ce0349439f07738f49a606cc: Deleting consensus metadata
I20260809 04:45:05.693536 9067 catalog_manager.cc:4841] TS 5be24e26ce0349439f07738f49a606cc (127.7.171.2:35229): tablet 666974058ef248b9af78bbeda6cb0511 (table test-table [id=5f6ed6c6e86f4b23a396f7a89c532218]) successfully deleted
I20260809 04:45:05.696251 9719 tablet_replica.cc:330] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871: stopping tablet replica
I20260809 04:45:05.696427 9719 raft_consensus.cc:2229] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:05.696569 9719 raft_consensus.cc:2258] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 04:45:05 dist-test-slave-1zqn krb5kdc[9033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:45:05.700747 9719 ts_tablet_manager.cc:1905] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:05.704617 9719 ts_tablet_manager.cc:1918] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:05.704771 9719 log.cc:1196] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/666974058ef248b9af78bbeda6cb0511
I20260809 04:45:05.705096 9719 ts_tablet_manager.cc:1939] T 666974058ef248b9af78bbeda6cb0511 P a43799bddc9947ed9bda1167eb307871: Deleting consensus metadata
I20260809 04:45:05.705662 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.692274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.7.171.1:41225)
0809 04:45:05.692390 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.692394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.693007 (+ 613us) server_negotiation.cc:407] Connection header received
0809 04:45:05.693189 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.693192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.693237 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.693285 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:05.694382 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.695171 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.696430 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.696769 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.696848 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.701088 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:05.701104 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:05.701123 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:05.701152 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:05.703316 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:05.703860 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:05.703867 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:05.703869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:05.703979 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:05.704331 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:05.704334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:05.704336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:05.704675 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:05.704856 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.705330 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.705476 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 04:45:05.706610 9296 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36342
I20260809 04:45:05.706633 9067 catalog_manager.cc:4841] TS a43799bddc9947ed9bda1167eb307871 (127.7.171.3:42811): tablet 666974058ef248b9af78bbeda6cb0511 (table test-table [id=5f6ed6c6e86f4b23a396f7a89c532218]) successfully deleted
I20260809 04:45:05.718590 9134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:05.709005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.7.171.62:42665)
0809 04:45:05.709155 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:05.709160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:05.709401 (+ 241us) server_negotiation.cc:407] Connection header received
0809 04:45:05.709581 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:05.709584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:05.709635 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:05.709714 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:05.710604 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.711427 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.712573 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:05.712918 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:05.713022 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:05.714780 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:05.714785 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:05.714788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:05.714817 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:05.716646 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:05.717162 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:05.717165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:05.717167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:05.717302 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:05.717628 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:05.717631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:05.717633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:05.717854 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:05.717982 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:05.718232 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 04:45:05.718416 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 04:45:05.719457 9081 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38434
I20260809 04:45:05.721119 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 9159
I20260809 04:45:05.735611 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 9314
I20260809 04:45:05.746140 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 9472
I20260809 04:45:05.753953 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 9046
2026-08-09T04:45:05Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3147 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:05 dist-test-slave-1zqn krb5kdc[9727](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:05 dist-test-slave-1zqn krb5kdc[9727](info): set up 2 sockets
Aug 09 04:45:05 dist-test-slave-1zqn krb5kdc[9727](info): commencing operation
krb5kdc: starting...
W20260809 04:45:06.858376 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:06 dist-test-slave-1zqn krb5kdc[9727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:06.990810 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43191
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:43191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:07.107838 9740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:07.108161 9740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:07.108242 9740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:07.112804 9740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:07.112882 9740 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:45:07.112912 9740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:07.112969 9740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:07.113020 9740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:07.113057 9740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:07.118351 9740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:43191
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:07.119438 9740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:07.120568 9740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:07.129045 9750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:07.129036 9748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:07.129036 9746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:07.129454 9740 server_base.cc:1034] running on GCE node
I20260809 04:45:07.130014 9740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:07.131126 9740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:07.132287 9740 hybrid_clock.cc:648] HybridClock initialized: now 1786250707132267 us; error 33 us; skew 500 ppm
Aug 09 04:45:07 dist-test-slave-1zqn krb5kdc[9727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250707, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:07.135178 9740 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:45:07.136102 9740 webserver.cc:460] Webserver started at http://127.7.171.62:33231/ using document root <none> and password file <none>
I20260809 04:45:07.136694 9740 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:07.136758 9740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:07.136989 9740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:07.139145 9740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "4f3e08f479194dd5af621c28ba6e8fa3"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "4ef11eb680798f1231aaf897cae1fdcc"
server_key_iv: "ca0148b045489a5c441535083bff24f8"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.139753 9740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "4f3e08f479194dd5af621c28ba6e8fa3"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "4ef11eb680798f1231aaf897cae1fdcc"
server_key_iv: "ca0148b045489a5c441535083bff24f8"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.143594 9740 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:45:07.146113 9757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:07.147394 9740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:07.147560 9740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "4f3e08f479194dd5af621c28ba6e8fa3"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "4ef11eb680798f1231aaf897cae1fdcc"
server_key_iv: "ca0148b045489a5c441535083bff24f8"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.147684 9740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:07.160841 9740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:07.164170 9740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:07.164376 9740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:07.172823 9740 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:43191
I20260809 04:45:07.172849 9828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:43191 every 8 connection(s)
I20260809 04:45:07.173789 9740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:07.176512 9829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:07.178069 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 9740
I20260809 04:45:07.178193 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:07.178450 7852 external_mini_cluster.cc:1428] Setting key 64db349caa53a5381b80d2bde0cbd7e6
I20260809 04:45:07.183290 9829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3: Bootstrap starting.
I20260809 04:45:07.185941 9829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:07.186968 9829 log.cc:824] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:45:07 dist-test-slave-1zqn krb5kdc[9727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:07.189363 9829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3: No bootstrap required, opened a new log
I20260809 04:45:07.194607 9829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e08f479194dd5af621c28ba6e8fa3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43191 } }
I20260809 04:45:07.194970 9829 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:07.195056 9829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3e08f479194dd5af621c28ba6e8fa3, State: Initialized, Role: FOLLOWER
I20260809 04:45:07.195616 9829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e08f479194dd5af621c28ba6e8fa3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43191 } }
I20260809 04:45:07.195827 9829 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:07.195920 9829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:07.196041 9829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:07.196858 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.179898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.7.171.62:43191)
0809 04:45:07.180365 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:07.180385 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:07.180428 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:45:07.181369 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:07.181393 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:07.181816 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:07.182189 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:07.182888 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.184332 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.185374 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.186084 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.186262 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.189718 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:07.189725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:07.189747 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:07.189788 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:07.193178 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.193686 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.193694 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.193701 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.193876 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.194217 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.194222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.194224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.194830 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:07.195199 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:07.195579 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 04:45:07.195924 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":154,"threads_started":1}
I20260809 04:45:07.197252 9829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e08f479194dd5af621c28ba6e8fa3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43191 } }
I20260809 04:45:07.197638 9829 leader_election.cc:302] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f3e08f479194dd5af621c28ba6e8fa3; no voters:
I20260809 04:45:07.198045 9829 leader_election.cc:288] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:07.198158 9834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:07.198361 9834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [term 1 LEADER]: Becoming Leader. State: Replica: 4f3e08f479194dd5af621c28ba6e8fa3, State: Running, Role: LEADER
I20260809 04:45:07.198719 9834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e08f479194dd5af621c28ba6e8fa3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43191 } }
I20260809 04:45:07.199589 9829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:07.200830 9836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f3e08f479194dd5af621c28ba6e8fa3. Latest consensus state: current_term: 1 leader_uuid: "4f3e08f479194dd5af621c28ba6e8fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e08f479194dd5af621c28ba6e8fa3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43191 } } }
I20260809 04:45:07.200908 9835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f3e08f479194dd5af621c28ba6e8fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f3e08f479194dd5af621c28ba6e8fa3" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43191 } } }
I20260809 04:45:07.201076 9835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:07.201110 9836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:07.201555 9844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:07.204731 9844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:07.211189 9844 catalog_manager.cc:1347] Generated new cluster ID: 69370efa1f3d453eb6e28bd386a13d58
I20260809 04:45:07.211269 9844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:07.220166 9844 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:07.221441 9844 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:07.235538 9844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3: Generated new TSK 0
I20260809 04:45:07.236445 9844 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:45:07.271845 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43191
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:07.387634 9860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:07.387897 9860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:07.387951 9860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:07.392036 9860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:07.392120 9860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:07.392161 9860 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:45:07.392176 9860 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:45:07.392208 9860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:07.397389 9860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.171.62:43191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:07.398468 9860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:07.399479 9860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:45:07.410244 9860 server_base.cc:1034] running on GCE node
W20260809 04:45:07.410286 9866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:07.410220 9865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:07.410214 9868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:07.410859 9860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:07.411576 9860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:07.412761 9860 hybrid_clock.cc:648] HybridClock initialized: now 1786250707412744 us; error 30 us; skew 500 ppm
Aug 09 04:45:07 dist-test-slave-1zqn krb5kdc[9727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250707, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:07.415712 9860 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:45:07.416731 9860 webserver.cc:460] Webserver started at http://127.7.171.1:42897/ using document root <none> and password file <none>
I20260809 04:45:07.417320 9860 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:07.417409 9860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:07.417651 9860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:07.420526 9860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "345d962f644249aab6f745b3a1b40af9"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "93c1106706ed375a59dd9fe66f4038f1"
server_key_iv: "fad59b98755ccba1cd7d7413e246e03b"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.421198 9860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "345d962f644249aab6f745b3a1b40af9"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "93c1106706ed375a59dd9fe66f4038f1"
server_key_iv: "fad59b98755ccba1cd7d7413e246e03b"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.425199 9860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:07.427779 9878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:07.429097 9860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 04:45:07.429229 9860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "345d962f644249aab6f745b3a1b40af9"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "93c1106706ed375a59dd9fe66f4038f1"
server_key_iv: "fad59b98755ccba1cd7d7413e246e03b"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.429348 9860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:07.469945 9860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:07.473291 9860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:07.473611 9860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:07.475147 9860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:07.475220 9860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:07.475286 9860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:07.475328 9860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:45:07 dist-test-slave-1zqn krb5kdc[9727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250707, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:07.486955 9860 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:35603
I20260809 04:45:07.486970 10006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:35603 every 8 connection(s)
I20260809 04:45:07.487946 9860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:07.490242 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 9860
I20260809 04:45:07.490268 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.476179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:53249 (local address 127.7.171.62:43191)
0809 04:45:07.476308 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:07.476312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:07.477489 (+ 1177us) server_negotiation.cc:407] Connection header received
0809 04:45:07.478787 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:07.478793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:07.478867 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:07.478975 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:07.480477 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.481399 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.483165 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.483407 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.483523 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.486401 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:07.486408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:07.486415 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:07.486452 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:07.488227 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.488845 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.488849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.488850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.488945 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.489286 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.489288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.489290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.489471 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:07.489545 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:07.490025 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 04:45:07.490141 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:45:07.490379 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:07.490692 7852 external_mini_cluster.cc:1428] Setting key b9eb3a4d2cc71d7073f7b5cc456a12db
I20260809 04:45:07.491354 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.476796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43191 (local address 127.7.171.1:53249)
0809 04:45:07.477338 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:07.477383 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:07.478507 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:07.479178 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:07.479190 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:07.479670 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:07.480323 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.480339 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.481545 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.481551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.483027 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.483041 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.483569 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.483573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.483758 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.484433 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:07.484450 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:07.486205 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:07.488344 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.488350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.488365 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.488717 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.489048 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.489051 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.489052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.489194 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.489556 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:07.489561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:07.489808 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 04:45:07.489986 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 04:45:07.490351 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":150,"threads_started":1}
I20260809 04:45:07.493021 10007 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43191
I20260809 04:45:07.493216 10007 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:07.493805 10007 heartbeater.cc:511] Master 127.7.171.62:43191 requested a full tablet report, sending...
I20260809 04:45:07.495491 9774 ts_manager.cc:194] Registered new tserver with Master: 345d962f644249aab6f745b3a1b40af9 (127.7.171.1:35603)
I20260809 04:45:07.496759 9774 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:53249
I20260809 04:45:07.503232 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.495321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:45773 (local address 127.7.171.62:43191)
0809 04:45:07.495437 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:07.495442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:07.495683 (+ 241us) server_negotiation.cc:407] Connection header received
0809 04:45:07.495858 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:07.495861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:07.495899 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:07.495979 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:07.496441 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.496985 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.498016 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.498424 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.498570 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.500005 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:07.500008 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:07.500010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:07.500031 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:07.501590 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.502082 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.502085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.502086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.502160 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.502463 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.502465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.502466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.502626 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:07.502701 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:07.503001 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 04:45:07.503118 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:45:07.503360 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.495275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43191 (local address 127.7.171.1:45773)
0809 04:45:07.495581 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:07.495619 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:07.495753 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:07.495995 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:07.495998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:07.496225 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:07.496327 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.496332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.497105 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.497108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.497886 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.497897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.498567 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.498570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.498678 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.499319 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:07.499330 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:07.499817 (+ 487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:07.501691 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.501695 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.501698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.501985 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.502254 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.502259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.502264 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.502372 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.502726 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:07.502732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:07.502836 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 04:45:07.502960 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 04:45:07.503116 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:45:07.523146 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43191
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:07.638778 10016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:07.639055 10016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:07.639106 10016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:07.643131 10016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:07.643198 10016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:07.643239 10016 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:45:07.643253 10016 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:45:07.643286 10016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:07.648638 10016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.171.62:43191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:07.649674 10016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:07.650653 10016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:07.660679 10022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:07.660679 10026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:07.660817 10023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:07.661155 10016 server_base.cc:1034] running on GCE node
I20260809 04:45:07.661579 10016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:07.662204 10016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:07.663398 10016 hybrid_clock.cc:648] HybridClock initialized: now 1786250707663375 us; error 32 us; skew 500 ppm
Aug 09 04:45:07 dist-test-slave-1zqn krb5kdc[9727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250707, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:07.666260 10016 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:45:07.667250 10016 webserver.cc:460] Webserver started at http://127.7.171.2:39181/ using document root <none> and password file <none>
I20260809 04:45:07.667867 10016 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:07.667960 10016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:07.668232 10016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:07.670373 10016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "03a1d643cf8643f88ea0c27a799840b6"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "265f4e263e2a7806f2a60fcb52e3700f"
server_key_iv: "fe3eb6db1dcbea1d0d3d9439452e4df9"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.670955 10016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "03a1d643cf8643f88ea0c27a799840b6"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "265f4e263e2a7806f2a60fcb52e3700f"
server_key_iv: "fe3eb6db1dcbea1d0d3d9439452e4df9"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.674804 10016 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 04:45:07.677289 10033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:07.678450 10016 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:45:07.678596 10016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "03a1d643cf8643f88ea0c27a799840b6"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "265f4e263e2a7806f2a60fcb52e3700f"
server_key_iv: "fe3eb6db1dcbea1d0d3d9439452e4df9"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.678716 10016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:07.690543 10016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:07.693969 10016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:07.694181 10016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:07.695580 10016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:07.695652 10016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:07.695724 10016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:07.695763 10016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:45:07 dist-test-slave-1zqn krb5kdc[9727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250707, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:07.707206 10016 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:36035
I20260809 04:45:07.707257 10164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:36035 every 8 connection(s)
I20260809 04:45:07.708336 10016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:07.710408 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.696477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:33537 (local address 127.7.171.62:43191)
0809 04:45:07.696604 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:07.696608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:07.697668 (+ 1060us) server_negotiation.cc:407] Connection header received
0809 04:45:07.698777 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:07.698783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:07.698855 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:07.698919 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:07.700676 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.701537 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.703478 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.703874 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.703985 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.706767 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:07.706771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:07.706774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:07.706809 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:07.708334 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.708880 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.708883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.708885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.708974 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.709333 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.709338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.709340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.709521 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:07.709625 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:07.710102 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 04:45:07.710235 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:45:07.710433 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 10016
I20260809 04:45:07.710563 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:07.710840 7852 external_mini_cluster.cc:1428] Setting key 0c75640c1400522cd88c25e178c95a25
I20260809 04:45:07.711160 10044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.697010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43191 (local address 127.7.171.2:33537)
0809 04:45:07.697519 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:07.697560 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:07.698498 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:07.699386 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:07.699395 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:07.699853 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:07.700522 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.700545 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.701685 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.701692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.703323 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.703338 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.704022 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.704026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.704241 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.704848 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:07.704864 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:07.706598 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:07.708453 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.708458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.708473 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.708739 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.709082 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.709085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.709087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.709236 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.709649 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:07.709655 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:07.709879 (+ 224us) client_negotiation.cc:769] Sending connection context
0809 04:45:07.710058 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 04:45:07.710366 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":141,"threads_started":1}
I20260809 04:45:07.712602 10165 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43191
I20260809 04:45:07.712782 10165 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:07.713349 10165 heartbeater.cc:511] Master 127.7.171.62:43191 requested a full tablet report, sending...
I20260809 04:45:07.714459 9774 ts_manager.cc:194] Registered new tserver with Master: 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2:36035)
I20260809 04:45:07.715032 9774 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:33537
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:45:07.722632 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.715560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:58261 (local address 127.7.171.62:43191)
0809 04:45:07.715671 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:07.715675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:07.715926 (+ 251us) server_negotiation.cc:407] Connection header received
0809 04:45:07.716035 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:07.716038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:07.716152 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:07.716248 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:07.716698 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.717200 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.718091 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.718308 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.718407 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.719506 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:07.719511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:07.719513 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:07.719537 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:07.721011 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.721442 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.721446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.721448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.721529 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.721837 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.721840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.721842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.722061 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:07.722151 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:07.722368 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 04:45:07.722460 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:45:07.722632 10177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.715501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43191 (local address 127.7.171.2:58261)
0809 04:45:07.715819 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:07.715834 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:07.715947 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:07.716265 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:07.716268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:07.716465 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:07.716583 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.716589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.717322 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.717325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.717957 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.717967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.718423 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.718427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.718522 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.719044 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:07.719056 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:07.719371 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:07.721142 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.721145 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.721147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.721342 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.721650 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.721653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.721655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.721736 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.722151 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:07.722153 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:07.722224 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 04:45:07.722307 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 04:45:07.722412 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":86,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:45:07.739430 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43191
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:07.858227 10180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:07.858466 10180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:07.858513 10180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:07.862442 10180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:07.862497 10180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:07.862535 10180 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:45:07.862555 10180 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:45:07.862586 10180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:07.867666 10180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.171.62:43191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:07.868726 10180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:07.869689 10180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:45:07.879526 10180 server_base.cc:1034] running on GCE node
W20260809 04:45:07.879467 10190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:07.879442 10186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:07.879446 10187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:07.880223 10180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:07.880807 10180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:07.881968 10180 hybrid_clock.cc:648] HybridClock initialized: now 1786250707881939 us; error 39 us; skew 500 ppm
Aug 09 04:45:07 dist-test-slave-1zqn krb5kdc[9727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250707, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:07.884877 10180 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:45:07.885782 10180 webserver.cc:460] Webserver started at http://127.7.171.3:33911/ using document root <none> and password file <none>
I20260809 04:45:07.886368 10180 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:07.886458 10180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:07.886693 10180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:07.888902 10180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "ec60354b057d4b6695870b8562a136b8"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "e6a9c5c0972bde4c638ccfba26a01621"
server_key_iv: "9180190968c6c7b4af6a4fb2d7025e97"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.889483 10180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "ec60354b057d4b6695870b8562a136b8"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "e6a9c5c0972bde4c638ccfba26a01621"
server_key_iv: "9180190968c6c7b4af6a4fb2d7025e97"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.893378 10180 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:07.895910 10197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:07.897217 10180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:45:07.897364 10180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "ec60354b057d4b6695870b8562a136b8"
format_stamp: "Formatted at 2026-08-09 04:45:07 on dist-test-slave-1zqn"
server_key: "e6a9c5c0972bde4c638ccfba26a01621"
server_key_iv: "9180190968c6c7b4af6a4fb2d7025e97"
server_key_version: "encryptionkey@0"
I20260809 04:45:07.897472 10180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:07.913833 10180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:07.917306 10180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:07.917510 10180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:07.919068 10180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:07.919143 10180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:07.919210 10180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:07.919255 10180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:45:07 dist-test-slave-1zqn krb5kdc[9727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250707, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:07.930332 10180 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:45351
I20260809 04:45:07.930712 10327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:45351 every 8 connection(s)
I20260809 04:45:07.931310 10180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:07.933790 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.919894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:40335 (local address 127.7.171.62:43191)
0809 04:45:07.920019 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:07.920023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:07.921158 (+ 1135us) server_negotiation.cc:407] Connection header received
0809 04:45:07.922312 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:07.922319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:07.922388 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:07.922456 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:07.924295 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.925220 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.926596 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.926830 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.926948 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.929850 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:07.929854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:07.929858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:07.929894 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:07.931453 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.932099 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.932105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.932108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.932209 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.932560 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.932563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.932564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.932811 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:07.932892 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:07.933526 (+ 634us) server_negotiation.cc:299] Negotiation successful
0809 04:45:07.933650 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 04:45:07.934603 10207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.920497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43191 (local address 127.7.171.3:40335)
0809 04:45:07.920983 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:07.921026 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:07.922035 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:07.922898 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:07.922906 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:07.923354 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:07.924109 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.924129 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.925370 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.925375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.926463 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.926473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.927002 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.927006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.927246 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.927864 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:07.927880 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:07.929679 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:07.931582 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.931588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.931602 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.931950 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.932296 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.932301 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.932303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.932449 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.932952 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:07.932958 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:07.933277 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 04:45:07.933466 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 04:45:07.933727 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":140,"threads_started":1}
I20260809 04:45:07.935882 10328 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43191
I20260809 04:45:07.936039 10328 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:07.936539 10328 heartbeater.cc:511] Master 127.7.171.62:43191 requested a full tablet report, sending...
I20260809 04:45:07.937664 9774 ts_manager.cc:194] Registered new tserver with Master: ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351)
I20260809 04:45:07.938411 9774 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:40335
I20260809 04:45:07.939457 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 10180
I20260809 04:45:07.939577 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:07.939869 7852 external_mini_cluster.cc:1428] Setting key cc83efeabd01f46649a6e5900c8a3c0b
I20260809 04:45:07.941952 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:45:07.946943 10335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.938236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43191 (local address 127.7.171.3:53007)
0809 04:45:07.938615 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:07.938627 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:07.938746 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:07.940157 (+ 1411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:07.940160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:07.940379 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:07.940482 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.940488 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.941239 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.941243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.941801 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.941808 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.942346 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.942349 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.942446 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.942985 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:07.942996 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:07.943331 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:07.945398 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.945401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.945403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.945614 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.945930 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:07.945932 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:07.945934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:07.946021 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:07.946492 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:07.946494 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:07.946574 (+ 80us) client_negotiation.cc:769] Sending connection context
0809 04:45:07.946653 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:45:07.946770 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":91,"threads_started":1}
I20260809 04:45:07.947115 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.938299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:53007 (local address 127.7.171.62:43191)
0809 04:45:07.938433 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:07.938437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:07.938718 (+ 281us) server_negotiation.cc:407] Connection header received
0809 04:45:07.938848 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:07.938852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:07.938913 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:07.939006 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:07.940607 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.941110 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.941921 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.942195 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.942340 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.943463 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:07.943467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:07.943470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:07.943494 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:07.945264 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.945717 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.945723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.945726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.945825 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.946109 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.946115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.946119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.946384 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:07.946487 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:07.946737 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 04:45:07.946879 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 04:45:07 dist-test-slave-1zqn krb5kdc[9727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:45:07 dist-test-slave-1zqn krb5kdc[9727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:07.970741 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.962275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.7.171.62:43191)
0809 04:45:07.962401 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:07.962404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:07.962416 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:07.962454 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:07.962456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:07.962494 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:07.962561 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:07.963289 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.963805 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.964718 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.964927 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.965006 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.967684 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:07.967688 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:07.967690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:07.967717 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:07.969125 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.969505 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.969508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.969509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.969601 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:07.969857 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:07.969860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:07.969862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:07.970084 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:07.970204 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:07.970395 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 04:45:07.970544 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:45:07.973659 9774 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:07.976795 9774 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:07.991807 10347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.985614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45351 (local address 127.0.0.1:59878)
0809 04:45:07.986103 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:07.986151 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:07.986286 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:07.987555 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:07.987560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:07.987584 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:07.987744 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.987752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.988844 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.988850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.990191 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.990202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.991216 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.991220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.991326 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.991369 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 04:45:07.991504 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 04:45:07.991632 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":33,"thread_start_us":88,"threads_started":1}
I20260809 04:45:07.992336 10348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.985799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:36035 (local address 127.0.0.1:50154)
0809 04:45:07.986529 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:07.986542 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:07.986628 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:07.987631 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:07.987635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:07.987662 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:07.987847 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.987857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.989378 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.989381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.990660 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.990670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.991821 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.991825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.991951 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.991987 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 04:45:07.992045 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 04:45:07.992141 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":81,"spinlock_wait_cycles":7424,"thread_start_us":133,"threads_started":1}
I20260809 04:45:07.992503 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.986354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:35603 (local address 127.0.0.1:55820)
0809 04:45:07.986934 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:07.986948 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:07.987047 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:07.987727 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:07.987730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:07.987753 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:07.987861 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.987867 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.989330 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.989337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.990666 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:07.990676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.992013 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.992020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:07.992135 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.992158 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:45:07.992242 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:45:07.992302 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":23,"spinlock_wait_cycles":1408,"thread_start_us":100,"threads_started":1}
I20260809 04:45:07.992555 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.986622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55820 (local address 127.7.171.1:35603)
0809 04:45:07.987214 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:07.987230 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:07.987252 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:45:07.987321 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:07.987326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:07.987597 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:07.987744 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:07.987961 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.989177 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.990842 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.991660 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.991791 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.991947 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:07.992289 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 04:45:07.992386 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":90,"threads_started":1}
I20260809 04:45:07.992918 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.985817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59878 (local address 127.7.171.3:45351)
0809 04:45:07.986771 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:07.986785 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:07.986811 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:45:07.986898 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:07.986904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:07.987388 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:07.987561 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:07.987877 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.988721 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.990348 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.991086 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.991205 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.991380 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:07.991544 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 04:45:07.991641 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851,"thread_start_us":139,"threads_started":1}
I20260809 04:45:07.993240 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:07.986577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50154 (local address 127.7.171.2:36035)
0809 04:45:07.987109 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:07.987123 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:07.987150 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:45:07.987234 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:07.987242 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:07.987445 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:07.988023 (+ 578us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:07.988204 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.989276 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.990791 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:07.991660 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:07.992672 (+ 1012us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:07.992879 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:07.992948 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 04:45:07.993041 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":123,"threads_started":1}
I20260809 04:45:07.995446 9932 tablet_service.cc:1467] Processing CreateTablet for tablet abf2971418094baa8fe1448631736f76 (DEFAULT_TABLE table=test-table [id=c945f57c7a41465e9b10fb20239fa801]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:07.995738 10090 tablet_service.cc:1467] Processing CreateTablet for tablet abf2971418094baa8fe1448631736f76 (DEFAULT_TABLE table=test-table [id=c945f57c7a41465e9b10fb20239fa801]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:07.995738 10253 tablet_service.cc:1467] Processing CreateTablet for tablet abf2971418094baa8fe1448631736f76 (DEFAULT_TABLE table=test-table [id=c945f57c7a41465e9b10fb20239fa801]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:07.996568 9932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abf2971418094baa8fe1448631736f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:07.996755 10090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abf2971418094baa8fe1448631736f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:07.996775 10253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abf2971418094baa8fe1448631736f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:08.003644 10353 tablet_bootstrap.cc:492] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9: Bootstrap starting.
I20260809 04:45:08.005306 10354 tablet_bootstrap.cc:492] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8: Bootstrap starting.
I20260809 04:45:08.005357 10355 tablet_bootstrap.cc:492] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: Bootstrap starting.
I20260809 04:45:08.006635 10353 tablet_bootstrap.cc:654] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:08.007231 10355 tablet_bootstrap.cc:654] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:08.007474 10354 tablet_bootstrap.cc:654] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:08.008134 10355 log.cc:824] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:08.008354 10353 log.cc:824] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:08.008405 10354 log.cc:824] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:08.010123 10355 tablet_bootstrap.cc:492] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: No bootstrap required, opened a new log
I20260809 04:45:08.010336 10353 tablet_bootstrap.cc:492] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9: No bootstrap required, opened a new log
I20260809 04:45:08.010349 10355 ts_tablet_manager.cc:1397] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:45:08.010383 10354 tablet_bootstrap.cc:492] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8: No bootstrap required, opened a new log
I20260809 04:45:08.010540 10353 ts_tablet_manager.cc:1397] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 04:45:08.010532 10354 ts_tablet_manager.cc:1397] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:45:08.013679 10355 raft_consensus.cc:348] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } }
I20260809 04:45:08.013952 10355 raft_consensus.cc:374] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:08.014014 10355 raft_consensus.cc:729] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a1d643cf8643f88ea0c27a799840b6, State: Initialized, Role: FOLLOWER
I20260809 04:45:08.014545 10355 consensus_queue.cc:260] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } }
I20260809 04:45:08.014551 10353 raft_consensus.cc:348] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } }
I20260809 04:45:08.014551 10354 raft_consensus.cc:348] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } }
I20260809 04:45:08.014812 10354 raft_consensus.cc:374] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:08.014812 10353 raft_consensus.cc:374] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:08.014865 10354 raft_consensus.cc:729] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec60354b057d4b6695870b8562a136b8, State: Initialized, Role: FOLLOWER
I20260809 04:45:08.014865 10353 raft_consensus.cc:729] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345d962f644249aab6f745b3a1b40af9, State: Initialized, Role: FOLLOWER
I20260809 04:45:08.015292 10353 consensus_queue.cc:260] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } }
I20260809 04:45:08.015292 10354 consensus_queue.cc:260] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } }
I20260809 04:45:08.015810 10355 ts_tablet_manager.cc:1428] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260809 04:45:08.016173 10165 heartbeater.cc:503] Master 127.7.171.62:43191 was elected leader, sending a full tablet report...
I20260809 04:45:08.016259 10353 ts_tablet_manager.cc:1428] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 04:45:08.016259 10354 ts_tablet_manager.cc:1428] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 04:45:08.016580 10328 heartbeater.cc:503] Master 127.7.171.62:43191 was elected leader, sending a full tablet report...
I20260809 04:45:08.017740 10007 heartbeater.cc:503] Master 127.7.171.62:43191 was elected leader, sending a full tablet report...
I20260809 04:45:08.054832 10359 raft_consensus.cc:482] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:08.055130 10359 raft_consensus.cc:504] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } }
I20260809 04:45:08.056666 10359 leader_election.cc:288] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351), 345d962f644249aab6f745b3a1b40af9 (127.7.171.1:35603)
I20260809 04:45:08.061252 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.057289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:49455 (local address 127.7.171.1:35603)
0809 04:45:08.057440 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:08.057445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:08.057742 (+ 297us) server_negotiation.cc:407] Connection header received
0809 04:45:08.057855 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:08.057859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:08.057912 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:08.057996 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:08.058290 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.058927 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.059992 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.060686 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.060777 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.060831 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:08.061042 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 04:45:08.061100 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:45:08.061273 10362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.057148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:35603 (local address 127.7.171.2:49455)
0809 04:45:08.057636 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:08.057650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:08.057758 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:08.058025 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:08.058027 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:08.058041 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:08.058148 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.058172 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.059041 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.059044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.059858 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.059867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.060801 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.060805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.060911 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.060937 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:45:08.061017 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:45:08.061085 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":139,"threads_started":1}
I20260809 04:45:08.061805 10044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.056883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45351 (local address 127.7.171.2:58327)
0809 04:45:08.057348 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:08.057367 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:08.057483 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:08.057853 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:08.057856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:08.057877 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:08.058084 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.058094 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.058878 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.058882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.060499 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.060511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.061345 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.061349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.061469 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.061498 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:45:08.061581 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 04:45:08.061655 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361}
I20260809 04:45:08.061874 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.057018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:58327 (local address 127.7.171.3:45351)
0809 04:45:08.057506 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:08.057510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:08.057532 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:45:08.057650 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:08.057653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:08.057724 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:08.057814 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:08.058218 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.058758 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.060623 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.061235 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.061366 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.061437 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:08.061677 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 04:45:08.061751 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382}
I20260809 04:45:08.062153 9953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abf2971418094baa8fe1448631736f76" candidate_uuid: "03a1d643cf8643f88ea0c27a799840b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345d962f644249aab6f745b3a1b40af9" is_pre_election: true
I20260809 04:45:08.062451 10276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abf2971418094baa8fe1448631736f76" candidate_uuid: "03a1d643cf8643f88ea0c27a799840b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60354b057d4b6695870b8562a136b8" is_pre_election: true
I20260809 04:45:08.062518 9953 raft_consensus.cc:2454] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03a1d643cf8643f88ea0c27a799840b6 in term 0.
I20260809 04:45:08.062772 10276 raft_consensus.cc:2454] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03a1d643cf8643f88ea0c27a799840b6 in term 0.
I20260809 04:45:08.063045 10036 leader_election.cc:302] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a1d643cf8643f88ea0c27a799840b6, 345d962f644249aab6f745b3a1b40af9; no voters:
I20260809 04:45:08.063344 10359 raft_consensus.cc:2781] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:08.063439 10359 raft_consensus.cc:482] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:08.063486 10359 raft_consensus.cc:3037] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:08.064615 10359 raft_consensus.cc:504] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } }
I20260809 04:45:08.064944 10359 leader_election.cc:288] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [CANDIDATE]: Term 1 election: Requested vote from peers ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351), 345d962f644249aab6f745b3a1b40af9 (127.7.171.1:35603)
I20260809 04:45:08.065304 10276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abf2971418094baa8fe1448631736f76" candidate_uuid: "03a1d643cf8643f88ea0c27a799840b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60354b057d4b6695870b8562a136b8"
I20260809 04:45:08.065331 9953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abf2971418094baa8fe1448631736f76" candidate_uuid: "03a1d643cf8643f88ea0c27a799840b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345d962f644249aab6f745b3a1b40af9"
I20260809 04:45:08.065436 10276 raft_consensus.cc:3037] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:08.065436 9953 raft_consensus.cc:3037] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:08.066728 9953 raft_consensus.cc:2454] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03a1d643cf8643f88ea0c27a799840b6 in term 1.
I20260809 04:45:08.066728 10276 raft_consensus.cc:2454] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03a1d643cf8643f88ea0c27a799840b6 in term 1.
I20260809 04:45:08.067049 10036 leader_election.cc:302] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a1d643cf8643f88ea0c27a799840b6, 345d962f644249aab6f745b3a1b40af9; no voters:
I20260809 04:45:08.067203 10359 raft_consensus.cc:2781] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:08.067426 10359 raft_consensus.cc:686] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 1 LEADER]: Becoming Leader. State: Replica: 03a1d643cf8643f88ea0c27a799840b6, State: Running, Role: LEADER
I20260809 04:45:08.067795 10359 consensus_queue.cc:237] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } }
I20260809 04:45:08.071296 9773 catalog_manager.cc:5495] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2). New cstate: current_term: 1 leader_uuid: "03a1d643cf8643f88ea0c27a799840b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } health_report { overall_health: UNKNOWN } } }
Aug 09 04:45:08 dist-test-slave-1zqn krb5kdc[9727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250707, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:08.115449 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.106045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42230 (local address 127.7.171.62:43191)
0809 04:45:08.106239 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:08.106246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:08.106421 (+ 175us) server_negotiation.cc:407] Connection header received
0809 04:45:08.106624 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:08.106627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:08.106673 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:08.106766 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:08.107577 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.108220 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.109097 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.109333 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.109450 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.111980 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:08.111984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:08.111986 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:08.112008 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:08.113523 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:08.114018 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:08.114025 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:08.114027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:08.114108 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:08.114406 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:08.114409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:08.114411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:08.114605 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:08.114719 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:08.115046 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 04:45:08.115267 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":73}
I20260809 04:45:08.115456 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.105952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43191 (local address 127.0.0.1:42230)
0809 04:45:08.106322 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:08.106336 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:08.106514 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:08.106785 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:08.106788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:08.107300 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:08.107429 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.107436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.108331 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.108338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.108969 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.108978 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.109459 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.109463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.109554 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.110229 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:08.110242 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:08.111818 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:08.113633 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:08.113647 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:08.113654 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:08.113906 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:08.114211 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:08.114215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:08.114217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:08.114305 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:08.114728 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:08.114739 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:08.114859 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 04:45:08.114995 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 04:45:08.115191 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":132,"threads_started":1}
I20260809 04:45:08.119668 9773 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:42230:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:45:08.120476 9773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:08.125846 10090 tablet_service.cc:1467] Processing CreateTablet for tablet 0f61cef0d35d4e41b9255654fe44466d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=61cc309bb79e4fd98aabf0e9af5e4248]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:45:08.125661 9932 tablet_service.cc:1467] Processing CreateTablet for tablet 0f61cef0d35d4e41b9255654fe44466d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=61cc309bb79e4fd98aabf0e9af5e4248]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:45:08.125876 10253 tablet_service.cc:1467] Processing CreateTablet for tablet 0f61cef0d35d4e41b9255654fe44466d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=61cc309bb79e4fd98aabf0e9af5e4248]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:45:08.126479 10090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f61cef0d35d4e41b9255654fe44466d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:08.126479 9932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f61cef0d35d4e41b9255654fe44466d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:08.126515 10253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f61cef0d35d4e41b9255654fe44466d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:08.129007 10354 tablet_bootstrap.cc:492] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: Bootstrap starting.
I20260809 04:45:08.129451 10355 tablet_bootstrap.cc:492] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6: Bootstrap starting.
I20260809 04:45:08.129451 10353 tablet_bootstrap.cc:492] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9: Bootstrap starting.
I20260809 04:45:08.130024 10354 tablet_bootstrap.cc:654] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:08.130467 10353 tablet_bootstrap.cc:654] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:08.130507 10355 tablet_bootstrap.cc:654] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:08.131860 10353 tablet_bootstrap.cc:492] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9: No bootstrap required, opened a new log
I20260809 04:45:08.131928 10354 tablet_bootstrap.cc:492] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: No bootstrap required, opened a new log
I20260809 04:45:08.131929 10355 tablet_bootstrap.cc:492] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6: No bootstrap required, opened a new log
I20260809 04:45:08.131976 10353 ts_tablet_manager.cc:1397] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:08.132017 10355 ts_tablet_manager.cc:1397] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:08.132017 10354 ts_tablet_manager.cc:1397] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:08.132534 10353 raft_consensus.cc:348] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.132536 10354 raft_consensus.cc:348] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.132536 10355 raft_consensus.cc:348] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.132655 10354 raft_consensus.cc:374] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:08.132654 10353 raft_consensus.cc:374] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:08.132691 10354 raft_consensus.cc:729] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec60354b057d4b6695870b8562a136b8, State: Initialized, Role: FOLLOWER
I20260809 04:45:08.132696 10353 raft_consensus.cc:729] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345d962f644249aab6f745b3a1b40af9, State: Initialized, Role: FOLLOWER
I20260809 04:45:08.132697 10355 raft_consensus.cc:374] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:08.132740 10355 raft_consensus.cc:729] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a1d643cf8643f88ea0c27a799840b6, State: Initialized, Role: FOLLOWER
I20260809 04:45:08.132830 10353 consensus_queue.cc:260] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.132853 10355 consensus_queue.cc:260] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.132844 10354 consensus_queue.cc:260] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.133133 10353 ts_tablet_manager.cc:1428] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 04:45:08.133149 10354 ts_tablet_manager.cc:1428] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:08.133165 10359 tablet_replica.cc:439] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } }
I20260809 04:45:08.133178 10361 tablet_replica.cc:439] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } }
I20260809 04:45:08.133296 10359 tablet_replica.cc:442] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:08.133322 10361 tablet_replica.cc:442] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:08.133366 10360 tablet_replica.cc:439] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } }
I20260809 04:45:08.133483 10360 tablet_replica.cc:442] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:08.134866 10355 ts_tablet_manager.cc:1428] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260809 04:45:08.147333 10360 raft_consensus.cc:482] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:08.147450 10360 raft_consensus.cc:504] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.148835 10360 leader_election.cc:288] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 345d962f644249aab6f745b3a1b40af9 (127.7.171.1:35603), 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2:36035)
I20260809 04:45:08.153862 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.149555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:37883 (local address 127.7.171.1:35603)
0809 04:45:08.149696 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:08.149701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:08.149714 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:45:08.149763 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:08.149765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:08.149798 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:08.149892 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:08.150130 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.150857 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.152553 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.153168 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.153285 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.153348 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:08.153662 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 04:45:08.153733 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:45:08.153862 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.149216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:40519 (local address 127.7.171.2:36035)
0809 04:45:08.149334 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:08.149338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:08.149353 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:08.149436 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:08.149439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:08.149490 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:08.149566 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:08.149881 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.150705 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.152553 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.153183 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.153285 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.153348 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:08.153662 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 04:45:08.153733 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:45:08.153955 10207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.149057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:36035 (local address 127.7.171.3:40519)
0809 04:45:08.149234 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:08.149252 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:08.149337 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:08.149599 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:08.149604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:08.149627 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:08.149765 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.149772 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.150823 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.150826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.152418 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.152431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.153304 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.153309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.153489 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.153549 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 04:45:08.153631 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 04:45:08.153785 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":134}
I20260809 04:45:08.153972 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.149084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:35603 (local address 127.7.171.3:37883)
0809 04:45:08.149592 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:08.149606 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:08.149689 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:08.149909 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:08.149911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:08.149928 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:08.150023 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.150028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.151059 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.151063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.152437 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.152443 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.153313 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.153316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.153492 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.153554 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 04:45:08.153631 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 04:45:08.153786 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":163,"thread_start_us":147,"threads_started":1}
I20260809 04:45:08.154438 9953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f61cef0d35d4e41b9255654fe44466d" candidate_uuid: "ec60354b057d4b6695870b8562a136b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345d962f644249aab6f745b3a1b40af9" is_pre_election: true
I20260809 04:45:08.154616 9953 raft_consensus.cc:2454] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec60354b057d4b6695870b8562a136b8 in term 0.
I20260809 04:45:08.154589 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f61cef0d35d4e41b9255654fe44466d" candidate_uuid: "ec60354b057d4b6695870b8562a136b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a1d643cf8643f88ea0c27a799840b6" is_pre_election: true
I20260809 04:45:08.154822 10112 raft_consensus.cc:2454] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec60354b057d4b6695870b8562a136b8 in term 0.
I20260809 04:45:08.155040 10199 leader_election.cc:302] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345d962f644249aab6f745b3a1b40af9, ec60354b057d4b6695870b8562a136b8; no voters:
I20260809 04:45:08.155351 10360 raft_consensus.cc:2781] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:08.155418 10360 raft_consensus.cc:482] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:08.155457 10360 raft_consensus.cc:3037] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:08.156490 10360 raft_consensus.cc:504] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.156848 10360 leader_election.cc:288] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [CANDIDATE]: Term 1 election: Requested vote from peers 345d962f644249aab6f745b3a1b40af9 (127.7.171.1:35603), 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2:36035)
I20260809 04:45:08.157226 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f61cef0d35d4e41b9255654fe44466d" candidate_uuid: "ec60354b057d4b6695870b8562a136b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a1d643cf8643f88ea0c27a799840b6"
I20260809 04:45:08.157241 9953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f61cef0d35d4e41b9255654fe44466d" candidate_uuid: "ec60354b057d4b6695870b8562a136b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345d962f644249aab6f745b3a1b40af9"
I20260809 04:45:08.157349 10112 raft_consensus.cc:3037] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:08.157358 9953 raft_consensus.cc:3037] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:08.158241 9953 raft_consensus.cc:2454] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec60354b057d4b6695870b8562a136b8 in term 1.
I20260809 04:45:08.158361 10112 raft_consensus.cc:2454] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec60354b057d4b6695870b8562a136b8 in term 1.
I20260809 04:45:08.158528 10199 leader_election.cc:302] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345d962f644249aab6f745b3a1b40af9, ec60354b057d4b6695870b8562a136b8; no voters:
I20260809 04:45:08.158697 10360 raft_consensus.cc:2781] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:08.158941 10360 raft_consensus.cc:686] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [term 1 LEADER]: Becoming Leader. State: Replica: ec60354b057d4b6695870b8562a136b8, State: Running, Role: LEADER
I20260809 04:45:08.159314 10360 consensus_queue.cc:237] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.161711 10375 tablet_replica.cc:439] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: TxnStatusTablet state changed. Reason: New leader ec60354b057d4b6695870b8562a136b8. Latest consensus state: current_term: 1 leader_uuid: "ec60354b057d4b6695870b8562a136b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } }
I20260809 04:45:08.161947 10375 tablet_replica.cc:442] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:45:08.162263 10377 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:45:08.162521 10377 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:45:08.162928 9772 catalog_manager.cc:5495] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec60354b057d4b6695870b8562a136b8 (127.7.171.3). New cstate: current_term: 1 leader_uuid: "ec60354b057d4b6695870b8562a136b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } health_report { overall_health: HEALTHY } } }
I20260809 04:45:08.180598 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.176822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45351 (local address 127.0.0.1:59888)
0809 04:45:08.177067 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:08.177083 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:08.177197 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:08.177530 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:08.177533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:08.177541 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:08.177652 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.177657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.178369 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.178374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.179003 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.179011 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.179449 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.179453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.179554 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.179613 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:08.180301 (+ 688us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:08.180318 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 04:45:08.180388 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 04:45:08.180477 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260809 04:45:08.180620 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.177066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.7.171.3:45351)
0809 04:45:08.177190 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:08.177195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:08.177210 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:08.177333 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:08.177341 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:08.177396 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:08.177515 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:08.177757 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.178263 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.179123 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.179343 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.179447 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.179759 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:08.180211 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:08.180328 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:08.180398 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 04:45:08.180451 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 04:45:08.182376 10329 tablet.cc:2369] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:45:08.186389 10112 raft_consensus.cc:1261] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Refusing update from remote peer ec60354b057d4b6695870b8562a136b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:08.186484 9953 raft_consensus.cc:1261] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Refusing update from remote peer ec60354b057d4b6695870b8562a136b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:08.187059 10375 consensus_queue.cc:1035] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:08.187229 10360 consensus_queue.cc:1035] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:08.190979 10361 tablet_replica.cc:439] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9: TxnStatusTablet state changed. Reason: New leader ec60354b057d4b6695870b8562a136b8. Latest consensus state: current_term: 1 leader_uuid: "ec60354b057d4b6695870b8562a136b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } }
I20260809 04:45:08.191135 10361 tablet_replica.cc:442] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:08.191138 10359 tablet_replica.cc:439] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6: TxnStatusTablet state changed. Reason: New leader ec60354b057d4b6695870b8562a136b8. Latest consensus state: current_term: 1 leader_uuid: "ec60354b057d4b6695870b8562a136b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } }
I20260809 04:45:08.191252 10359 tablet_replica.cc:442] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:08.193677 10375 tablet_replica.cc:439] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec60354b057d4b6695870b8562a136b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } }
I20260809 04:45:08.193830 10375 tablet_replica.cc:442] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:45:08.195333 10361 tablet_replica.cc:439] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec60354b057d4b6695870b8562a136b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } }
I20260809 04:45:08.195444 10361 tablet_replica.cc:442] T 0f61cef0d35d4e41b9255654fe44466d P 345d962f644249aab6f745b3a1b40af9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:08.196028 10360 tablet_replica.cc:439] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec60354b057d4b6695870b8562a136b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } }
I20260809 04:45:08.196161 10360 tablet_replica.cc:442] T 0f61cef0d35d4e41b9255654fe44466d P ec60354b057d4b6695870b8562a136b8: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:45:08.198081 10359 tablet_replica.cc:439] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec60354b057d4b6695870b8562a136b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } } }
I20260809 04:45:08.198203 10359 tablet_replica.cc:442] T 0f61cef0d35d4e41b9255654fe44466d P 03a1d643cf8643f88ea0c27a799840b6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:08.198328 10381 mvcc.cc:205] Tried to move back new op lower bound from 7316482900724281344 to 7316482900623446016. Current Snapshot: MvccSnapshot[applied={T|T < 7316482900724281344}]
I20260809 04:45:08.204659 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.200094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50158 (local address 127.7.171.2:36035)
0809 04:45:08.200199 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:08.200204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:08.200266 (+ 62us) server_negotiation.cc:407] Connection header received
0809 04:45:08.200487 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:08.200490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:08.200553 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:08.200664 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:08.201035 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.201871 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.202905 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.203269 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.203375 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.203617 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:08.204193 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:08.204311 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:08.204457 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 04:45:08.204521 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 04:45:08.209702 10167 tablet.cc:2369] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:45:08.216936 10177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.212832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45351 (local address 127.7.171.2:39077)
0809 04:45:08.212971 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:08.212987 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:08.213071 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:08.213317 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:08.213320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:08.213327 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:08.213480 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.213487 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.214223 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.214227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.215197 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.215208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.216000 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.216004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.216175 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.216259 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:08.216670 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:08.216676 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:45:08.216734 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 04:45:08.216810 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 04:45:08.216952 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.212986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:39077 (local address 127.7.171.3:45351)
0809 04:45:08.213108 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:08.213112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:08.213125 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:45:08.213171 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:08.213173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:08.213216 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:08.213286 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:08.213614 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.214125 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.215350 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.215755 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.215898 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.216466 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:08.216564 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:08.216639 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:08.216763 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 04:45:08.216820 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:45:08.224397 9953 raft_consensus.cc:1261] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Refusing update from remote peer 03a1d643cf8643f88ea0c27a799840b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:08.224687 10276 raft_consensus.cc:1261] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Refusing update from remote peer 03a1d643cf8643f88ea0c27a799840b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:08.225291 10363 consensus_queue.cc:1035] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:08.225534 10359 consensus_queue.cc:1035] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:08.232100 10380 mvcc.cc:205] Tried to move back new op lower bound from 7316482900882817024 to 7316482900248813568. Current Snapshot: MvccSnapshot[applied={T|T < 7316482900882817024}]
I20260809 04:45:08.258170 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.254312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:57221 (local address 127.7.171.2:36035)
0809 04:45:08.254477 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:08.254482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:08.254498 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:45:08.254575 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:08.254581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:08.254632 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:08.254733 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:08.255011 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.255800 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.257079 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.257293 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.257369 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.257673 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:08.257787 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:08.257866 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:08.257970 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 04:45:08.258031 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:45:08.258174 10335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:08.254241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:36035 (local address 127.7.171.3:57221)
0809 04:45:08.254411 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:08.254425 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:08.254500 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:08.254746 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:08.254749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:08.254757 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:08.254881 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.254888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.255936 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.255939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.256943 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:08.256954 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:08.257393 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:08.257397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:08.257494 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:08.257550 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:08.257878 (+ 328us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:08.257885 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 04:45:08.257944 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 04:45:08.258022 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 04:45:08.281039 9772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:08.281240 9772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:08.284046 9772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3: Sending DeleteTablet for 3 replicas of tablet abf2971418094baa8fe1448631736f76
I20260809 04:45:08.285161 10090 tablet_service.cc:1514] Processing DeleteTablet for tablet abf2971418094baa8fe1448631736f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:08 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:50154
I20260809 04:45:08.285727 10402 tablet_replica.cc:330] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: stopping tablet replica
I20260809 04:45:08.286135 10402 raft_consensus.cc:2229] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:08.286391 10253 tablet_service.cc:1514] Processing DeleteTablet for tablet abf2971418094baa8fe1448631736f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:08 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:59878
I20260809 04:45:08.286687 10402 raft_consensus.cc:2258] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:08.286926 9772 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:08.287362 9772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:08.288431 10402 ts_tablet_manager.cc:1905] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:08.288371 9932 tablet_service.cc:1514] Processing DeleteTablet for tablet abf2971418094baa8fe1448631736f76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:08 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:55820
I20260809 04:45:08.291891 10402 ts_tablet_manager.cc:1918] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:08.292088 10402 log.cc:1196] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/abf2971418094baa8fe1448631736f76
I20260809 04:45:08.292523 10402 ts_tablet_manager.cc:1939] T abf2971418094baa8fe1448631736f76 P 03a1d643cf8643f88ea0c27a799840b6: Deleting consensus metadata
I20260809 04:45:08.293489 10090 tablet_service.cc:1467] Processing CreateTablet for tablet 06548647222946d99fe9a68f7c36673f (DEFAULT_TABLE table=test-table [id=ac961e96969048209e1a9f5b9a16554d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:08.293749 10090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06548647222946d99fe9a68f7c36673f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:08.295966 10355 tablet_bootstrap.cc:492] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6: Bootstrap starting.
I20260809 04:45:08.296278 10404 tablet_replica.cc:330] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9: stopping tablet replica
I20260809 04:45:08.296662 10404 raft_consensus.cc:2229] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:08.296901 10404 raft_consensus.cc:2258] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:08.296991 10355 tablet_bootstrap.cc:654] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:08.297858 9758 catalog_manager.cc:4841] TS 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2:36035): tablet abf2971418094baa8fe1448631736f76 (table test-table [id=c945f57c7a41465e9b10fb20239fa801]) successfully deleted
I20260809 04:45:08.298682 10355 tablet_bootstrap.cc:492] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6: No bootstrap required, opened a new log
I20260809 04:45:08.298774 10355 ts_tablet_manager.cc:1397] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 04:45:08.299217 10355 raft_consensus.cc:348] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.299336 10355 raft_consensus.cc:374] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:08.299373 10355 raft_consensus.cc:729] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a1d643cf8643f88ea0c27a799840b6, State: Initialized, Role: FOLLOWER
I20260809 04:45:08.299501 10355 consensus_queue.cc:260] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.299767 10355 ts_tablet_manager.cc:1428] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260809 04:45:08.300638 10253 tablet_service.cc:1467] Processing CreateTablet for tablet 06548647222946d99fe9a68f7c36673f (DEFAULT_TABLE table=test-table [id=ac961e96969048209e1a9f5b9a16554d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:08.300987 10253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06548647222946d99fe9a68f7c36673f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:08.301762 10404 ts_tablet_manager.cc:1905] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:08.303544 10354 tablet_bootstrap.cc:492] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8: Bootstrap starting.
I20260809 04:45:08.304216 9932 tablet_service.cc:1467] Processing CreateTablet for tablet 06548647222946d99fe9a68f7c36673f (DEFAULT_TABLE table=test-table [id=ac961e96969048209e1a9f5b9a16554d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:08.304643 10354 tablet_bootstrap.cc:654] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:08.305781 10403 tablet_replica.cc:330] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8: stopping tablet replica
I20260809 04:45:08.306102 10403 raft_consensus.cc:2229] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:08.306267 10403 raft_consensus.cc:2258] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:08.308445 10403 ts_tablet_manager.cc:1905] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:08.308836 10354 tablet_bootstrap.cc:492] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8: No bootstrap required, opened a new log
I20260809 04:45:08.308920 10354 ts_tablet_manager.cc:1397] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 04:45:08.309391 10354 raft_consensus.cc:348] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.309506 10354 raft_consensus.cc:374] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:08.309540 10354 raft_consensus.cc:729] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec60354b057d4b6695870b8562a136b8, State: Initialized, Role: FOLLOWER
I20260809 04:45:08.309666 10354 consensus_queue.cc:260] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.309924 10354 ts_tablet_manager.cc:1428] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 04:45:08.310141 9932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06548647222946d99fe9a68f7c36673f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:08.311033 10404 ts_tablet_manager.cc:1918] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:08.311151 10404 log.cc:1196] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/wal/wals/abf2971418094baa8fe1448631736f76
I20260809 04:45:08.311518 10404 ts_tablet_manager.cc:1939] T abf2971418094baa8fe1448631736f76 P 345d962f644249aab6f745b3a1b40af9: Deleting consensus metadata
I20260809 04:45:08.312593 10353 tablet_bootstrap.cc:492] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9: Bootstrap starting.
I20260809 04:45:08.313585 10353 tablet_bootstrap.cc:654] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:08.314590 9759 catalog_manager.cc:4841] TS 345d962f644249aab6f745b3a1b40af9 (127.7.171.1:35603): tablet abf2971418094baa8fe1448631736f76 (table test-table [id=c945f57c7a41465e9b10fb20239fa801]) successfully deleted
I20260809 04:45:08.315618 10403 ts_tablet_manager.cc:1918] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:08.315778 10403 log.cc:1196] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/abf2971418094baa8fe1448631736f76
I20260809 04:45:08.316306 10403 ts_tablet_manager.cc:1939] T abf2971418094baa8fe1448631736f76 P ec60354b057d4b6695870b8562a136b8: Deleting consensus metadata
I20260809 04:45:08.317525 10353 tablet_bootstrap.cc:492] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9: No bootstrap required, opened a new log
I20260809 04:45:08.317616 10353 ts_tablet_manager.cc:1397] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260809 04:45:08.318068 10353 raft_consensus.cc:348] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.318189 10353 raft_consensus.cc:374] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:08.318220 10353 raft_consensus.cc:729] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345d962f644249aab6f745b3a1b40af9, State: Initialized, Role: FOLLOWER
I20260809 04:45:08.318341 10353 consensus_queue.cc:260] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.318606 10353 ts_tablet_manager.cc:1428] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:08.319736 9759 catalog_manager.cc:4841] TS ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351): tablet abf2971418094baa8fe1448631736f76 (table test-table [id=c945f57c7a41465e9b10fb20239fa801]) successfully deleted
I20260809 04:45:08.333377 10363 raft_consensus.cc:482] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:08.333565 10363 raft_consensus.cc:504] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.334079 10363 leader_election.cc:288] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 345d962f644249aab6f745b3a1b40af9 (127.7.171.1:35603), ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351)
I20260809 04:45:08.334627 10276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06548647222946d99fe9a68f7c36673f" candidate_uuid: "03a1d643cf8643f88ea0c27a799840b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60354b057d4b6695870b8562a136b8" is_pre_election: true
I20260809 04:45:08.334636 9953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06548647222946d99fe9a68f7c36673f" candidate_uuid: "03a1d643cf8643f88ea0c27a799840b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345d962f644249aab6f745b3a1b40af9" is_pre_election: true
I20260809 04:45:08.334792 10276 raft_consensus.cc:2454] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03a1d643cf8643f88ea0c27a799840b6 in term 0.
I20260809 04:45:08.334800 9953 raft_consensus.cc:2454] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03a1d643cf8643f88ea0c27a799840b6 in term 0.
I20260809 04:45:08.335165 10036 leader_election.cc:302] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a1d643cf8643f88ea0c27a799840b6, ec60354b057d4b6695870b8562a136b8; no voters:
I20260809 04:45:08.335577 10363 raft_consensus.cc:2781] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:08.335645 10363 raft_consensus.cc:482] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:08.335700 10363 raft_consensus.cc:3037] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:08.337167 10363 raft_consensus.cc:504] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.337555 10363 leader_election.cc:288] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [CANDIDATE]: Term 1 election: Requested vote from peers 345d962f644249aab6f745b3a1b40af9 (127.7.171.1:35603), ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351)
I20260809 04:45:08.337946 10276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06548647222946d99fe9a68f7c36673f" candidate_uuid: "03a1d643cf8643f88ea0c27a799840b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60354b057d4b6695870b8562a136b8"
I20260809 04:45:08.338078 10276 raft_consensus.cc:3037] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:08.338092 9953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06548647222946d99fe9a68f7c36673f" candidate_uuid: "03a1d643cf8643f88ea0c27a799840b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345d962f644249aab6f745b3a1b40af9"
I20260809 04:45:08.338215 9953 raft_consensus.cc:3037] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:08.339212 10276 raft_consensus.cc:2454] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03a1d643cf8643f88ea0c27a799840b6 in term 1.
I20260809 04:45:08.339571 9953 raft_consensus.cc:2454] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03a1d643cf8643f88ea0c27a799840b6 in term 1.
I20260809 04:45:08.339941 10036 leader_election.cc:302] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a1d643cf8643f88ea0c27a799840b6, ec60354b057d4b6695870b8562a136b8; no voters:
I20260809 04:45:08.340178 10363 raft_consensus.cc:2781] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:08.340308 10363 raft_consensus.cc:686] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 1 LEADER]: Becoming Leader. State: Replica: 03a1d643cf8643f88ea0c27a799840b6, State: Running, Role: LEADER
I20260809 04:45:08.340461 10363 consensus_queue.cc:237] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:08.342388 9772 catalog_manager.cc:5495] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2). New cstate: current_term: 1 leader_uuid: "03a1d643cf8643f88ea0c27a799840b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260809 04:45:08.369752 10276 raft_consensus.cc:1261] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Refusing update from remote peer 03a1d643cf8643f88ea0c27a799840b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:08.369921 9952 raft_consensus.cc:1261] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Refusing update from remote peer 03a1d643cf8643f88ea0c27a799840b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:08.370272 10363 consensus_queue.cc:1035] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:08.370430 10359 consensus_queue.cc:1035] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:08.374823 10382 mvcc.cc:205] Tried to move back new op lower bound from 7316482901479178240 to 7316482901364068352. Current Snapshot: MvccSnapshot[applied={T|T < 7316482901479178240}]
W20260809 04:45:08.388564 10381 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 04:45:09.941963 9772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:09.942129 9772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:09.944507 9772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3: Sending DeleteTablet for 3 replicas of tablet 06548647222946d99fe9a68f7c36673f
I20260809 04:45:09.945052 10090 tablet_service.cc:1514] Processing DeleteTablet for tablet 06548647222946d99fe9a68f7c36673f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:09 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:50154
I20260809 04:45:09.945152 10253 tablet_service.cc:1514] Processing DeleteTablet for tablet 06548647222946d99fe9a68f7c36673f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:09 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:59878
I20260809 04:45:09.945364 9932 tablet_service.cc:1514] Processing DeleteTablet for tablet 06548647222946d99fe9a68f7c36673f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:09 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:55820
I20260809 04:45:09.945554 10443 tablet_replica.cc:330] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8: stopping tablet replica
I20260809 04:45:09.945716 10443 raft_consensus.cc:2229] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:09.945825 10443 raft_consensus.cc:2258] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:09.946389 10443 ts_tablet_manager.cc:1905] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:09.948043 10442 tablet_replica.cc:330] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6: stopping tablet replica
I20260809 04:45:09.948205 10442 raft_consensus.cc:2229] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:09.948387 10442 raft_consensus.cc:2258] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:09.949156 10442 ts_tablet_manager.cc:1905] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:09.949254 10443 ts_tablet_manager.cc:1918] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:45:09.949330 10443 log.cc:1196] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/06548647222946d99fe9a68f7c36673f
I20260809 04:45:09.949617 10443 ts_tablet_manager.cc:1939] T 06548647222946d99fe9a68f7c36673f P ec60354b057d4b6695870b8562a136b8: Deleting consensus metadata
I20260809 04:45:09.950263 9759 catalog_manager.cc:4841] TS ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351): tablet 06548647222946d99fe9a68f7c36673f (table test-table [id=ac961e96969048209e1a9f5b9a16554d]) successfully deleted
I20260809 04:45:09.951731 10442 ts_tablet_manager.cc:1918] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:45:09.951797 10442 log.cc:1196] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/06548647222946d99fe9a68f7c36673f
I20260809 04:45:09.952009 10442 ts_tablet_manager.cc:1939] T 06548647222946d99fe9a68f7c36673f P 03a1d643cf8643f88ea0c27a799840b6: Deleting consensus metadata
I20260809 04:45:09.952750 9758 catalog_manager.cc:4841] TS 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2:36035): tablet 06548647222946d99fe9a68f7c36673f (table test-table [id=ac961e96969048209e1a9f5b9a16554d]) successfully deleted
I20260809 04:45:09.955077 10444 tablet_replica.cc:330] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9: stopping tablet replica
I20260809 04:45:09.955229 10444 raft_consensus.cc:2229] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:09.955353 10444 raft_consensus.cc:2258] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:09.955917 10444 ts_tablet_manager.cc:1905] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:09.959228 10444 ts_tablet_manager.cc:1918] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:45:09.959326 10444 log.cc:1196] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/wal/wals/06548647222946d99fe9a68f7c36673f
I20260809 04:45:09.959622 10444 ts_tablet_manager.cc:1939] T 06548647222946d99fe9a68f7c36673f P 345d962f644249aab6f745b3a1b40af9: Deleting consensus metadata
I20260809 04:45:09.960390 9759 catalog_manager.cc:4841] TS 345d962f644249aab6f745b3a1b40af9 (127.7.171.1:35603): tablet 06548647222946d99fe9a68f7c36673f (table test-table [id=ac961e96969048209e1a9f5b9a16554d]) successfully deleted
Aug 09 04:45:09 dist-test-slave-1zqn krb5kdc[9727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250709, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:45:09 dist-test-slave-1zqn krb5kdc[9727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250709, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:09.984651 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:09.968868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.7.171.62:43191)
0809 04:45:09.969219 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:09.969228 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:09.969243 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:09.969368 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:09.969371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:09.969425 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:09.969520 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:09.970237 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:09.970814 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:09.971688 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:09.971938 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:09.972144 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:09.980872 (+ 8728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:09.980876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:09.980878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:09.980909 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:09.982639 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:09.983134 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:09.983140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:09.983142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:09.983230 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:09.983529 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:09.983531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:09.983532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:09.983733 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:09.983836 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:09.984243 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 04:45:09.984397 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
I20260809 04:45:09.987394 9772 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:09.987911 9772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:09.992779 9932 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0adeb23c51440ba4fd26732174effe (DEFAULT_TABLE table=test-table [id=bd0003ce297a4a009edba7a4a303f142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:09.992964 10090 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0adeb23c51440ba4fd26732174effe (DEFAULT_TABLE table=test-table [id=bd0003ce297a4a009edba7a4a303f142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:09.993125 9932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0adeb23c51440ba4fd26732174effe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:09.993099 10253 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0adeb23c51440ba4fd26732174effe (DEFAULT_TABLE table=test-table [id=bd0003ce297a4a009edba7a4a303f142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:09.993257 10090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0adeb23c51440ba4fd26732174effe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:09.993378 10253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0adeb23c51440ba4fd26732174effe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:09.995301 10453 tablet_bootstrap.cc:492] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9: Bootstrap starting.
I20260809 04:45:09.995987 10454 tablet_bootstrap.cc:492] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8: Bootstrap starting.
I20260809 04:45:09.996187 10455 tablet_bootstrap.cc:492] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6: Bootstrap starting.
I20260809 04:45:09.996831 10453 tablet_bootstrap.cc:654] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:09.996907 10454 tablet_bootstrap.cc:654] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:09.997022 10455 tablet_bootstrap.cc:654] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:09.998035 10455 tablet_bootstrap.cc:492] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6: No bootstrap required, opened a new log
I20260809 04:45:09.998081 10453 tablet_bootstrap.cc:492] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9: No bootstrap required, opened a new log
I20260809 04:45:09.998155 10455 ts_tablet_manager.cc:1397] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 04:45:09.998163 10453 ts_tablet_manager.cc:1397] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:09.998687 10454 tablet_bootstrap.cc:492] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8: No bootstrap required, opened a new log
I20260809 04:45:09.998610 10455 raft_consensus.cc:348] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:09.998615 10453 raft_consensus.cc:348] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:09.998734 10455 raft_consensus.cc:374] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:09.998751 10453 raft_consensus.cc:374] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:09.998778 10455 raft_consensus.cc:729] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a1d643cf8643f88ea0c27a799840b6, State: Initialized, Role: FOLLOWER
I20260809 04:45:09.998791 10453 raft_consensus.cc:729] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345d962f644249aab6f745b3a1b40af9, State: Initialized, Role: FOLLOWER
I20260809 04:45:09.998925 10454 ts_tablet_manager.cc:1397] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:09.998907 10455 consensus_queue.cc:260] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:09.998919 10453 consensus_queue.cc:260] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:09.999195 10455 ts_tablet_manager.cc:1428] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 04:45:09.999356 10453 ts_tablet_manager.cc:1428] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:09.999392 10454 raft_consensus.cc:348] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:09.999516 10454 raft_consensus.cc:374] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:09.999548 10454 raft_consensus.cc:729] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec60354b057d4b6695870b8562a136b8, State: Initialized, Role: FOLLOWER
I20260809 04:45:09.999680 10454 consensus_queue.cc:260] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.000011 10454 ts_tablet_manager.cc:1428] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:10.099066 10459 raft_consensus.cc:482] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:10.099253 10459 raft_consensus.cc:504] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.100644 10459 leader_election.cc:288] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2:36035), ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351)
I20260809 04:45:10.104830 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:10.100829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45351 (local address 127.7.171.1:58315)
0809 04:45:10.101371 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:10.101386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:10.101541 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:10.101837 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:10.101840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:10.101857 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:10.102005 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:10.102014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.102825 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.102829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:10.103665 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:10.103674 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.104349 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.104353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:10.104453 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:10.104480 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:45:10.104572 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:45:10.104643 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":55,"spinlock_wait_cycles":3712,"thread_start_us":301,"threads_started":1}
I20260809 04:45:10.104815 10465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:10.101017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:58315 (local address 127.7.171.3:45351)
0809 04:45:10.101492 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:10.101502 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:10.101517 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:10.101678 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:10.101682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:10.101731 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:10.101827 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:10.102125 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.102716 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.103786 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.104227 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.104343 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:10.104412 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:10.104588 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 04:45:10.104664 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":159,"threads_started":1}
I20260809 04:45:10.104863 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:10.101041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:45735 (local address 127.7.171.2:36035)
0809 04:45:10.101436 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:10.101445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:10.101461 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:45:10.101649 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:10.101653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:10.101704 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:10.101796 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:10.102092 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.102733 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.103801 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.104285 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.104398 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:10.104460 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:10.104644 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 04:45:10.104717 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":97,"threads_started":1}
I20260809 04:45:10.104877 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:10.100829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:36035 (local address 127.7.171.1:45735)
0809 04:45:10.101362 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:10.101379 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:10.101529 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:10.101810 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:10.101813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:10.101829 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:10.101967 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:10.101975 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.102875 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.102878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:10.103674 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:10.103683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.104405 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.104408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:10.104508 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:10.104535 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:45:10.104627 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:45:10.104696 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":1,"spinlock_wait_cycles":41088,"thread_start_us":259,"threads_started":1}
I20260809 04:45:10.105558 10276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0adeb23c51440ba4fd26732174effe" candidate_uuid: "345d962f644249aab6f745b3a1b40af9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60354b057d4b6695870b8562a136b8" is_pre_election: true
I20260809 04:45:10.105576 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0adeb23c51440ba4fd26732174effe" candidate_uuid: "345d962f644249aab6f745b3a1b40af9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a1d643cf8643f88ea0c27a799840b6" is_pre_election: true
I20260809 04:45:10.105717 10276 raft_consensus.cc:2454] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345d962f644249aab6f745b3a1b40af9 in term 0.
I20260809 04:45:10.105722 10112 raft_consensus.cc:2454] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345d962f644249aab6f745b3a1b40af9 in term 0.
I20260809 04:45:10.106119 9880 leader_election.cc:302] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345d962f644249aab6f745b3a1b40af9, ec60354b057d4b6695870b8562a136b8; no voters:
I20260809 04:45:10.106393 10459 raft_consensus.cc:2781] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:10.106463 10459 raft_consensus.cc:482] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:10.106503 10459 raft_consensus.cc:3037] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:10.107502 10459 raft_consensus.cc:504] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.107838 10459 leader_election.cc:288] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [CANDIDATE]: Term 1 election: Requested vote from peers 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2:36035), ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351)
I20260809 04:45:10.108168 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0adeb23c51440ba4fd26732174effe" candidate_uuid: "345d962f644249aab6f745b3a1b40af9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a1d643cf8643f88ea0c27a799840b6"
I20260809 04:45:10.108282 10112 raft_consensus.cc:3037] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:10.108266 10276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0adeb23c51440ba4fd26732174effe" candidate_uuid: "345d962f644249aab6f745b3a1b40af9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60354b057d4b6695870b8562a136b8"
I20260809 04:45:10.108389 10276 raft_consensus.cc:3037] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:10.109222 10112 raft_consensus.cc:2454] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 345d962f644249aab6f745b3a1b40af9 in term 1.
I20260809 04:45:10.109377 10276 raft_consensus.cc:2454] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 345d962f644249aab6f745b3a1b40af9 in term 1.
I20260809 04:45:10.109546 9879 leader_election.cc:302] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a1d643cf8643f88ea0c27a799840b6, 345d962f644249aab6f745b3a1b40af9; no voters:
I20260809 04:45:10.109726 10459 raft_consensus.cc:2781] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:10.109947 10459 raft_consensus.cc:686] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 1 LEADER]: Becoming Leader. State: Replica: 345d962f644249aab6f745b3a1b40af9, State: Running, Role: LEADER
I20260809 04:45:10.110209 10459 consensus_queue.cc:237] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.112596 9773 catalog_manager.cc:5495] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 345d962f644249aab6f745b3a1b40af9 (127.7.171.1). New cstate: current_term: 1 leader_uuid: "345d962f644249aab6f745b3a1b40af9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260809 04:45:10.130932 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:10.127246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55826 (local address 127.7.171.1:35603)
0809 04:45:10.127491 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:10.127498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:10.127510 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:10.127578 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:10.127581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:10.127632 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:10.127746 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:10.128043 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.128698 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.129545 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.129805 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.129895 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:10.130128 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:10.130596 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:10.130687 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:10.130757 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 04:45:10.130807 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":111,"threads_started":1}
I20260809 04:45:10.141820 10474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:10.138173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45351 (local address 127.7.171.1:40609)
0809 04:45:10.138506 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:10.138521 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:10.138655 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:10.138920 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:10.138923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:10.138931 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:10.139061 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:10.139066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.139789 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.139793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:10.140608 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:10.140618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.141070 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.141073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:10.141171 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:10.141215 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:10.141529 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:10.141537 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:45:10.141598 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 04:45:10.141677 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":105,"threads_started":1}
I20260809 04:45:10.141806 10465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:10.138300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:40609 (local address 127.7.171.3:45351)
0809 04:45:10.138468 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:10.138475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:10.138596 (+ 121us) server_negotiation.cc:407] Connection header received
0809 04:45:10.138767 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:10.138770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:10.138817 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:10.138894 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:10.139163 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.139683 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.140733 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.140953 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.141056 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:10.141360 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:10.141430 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:10.141509 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:10.141608 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 04:45:10.141661 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:45:10.147742 10276 raft_consensus.cc:1261] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Refusing update from remote peer 345d962f644249aab6f745b3a1b40af9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:10.147766 10112 raft_consensus.cc:1261] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Refusing update from remote peer 345d962f644249aab6f745b3a1b40af9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:10.148264 10459 consensus_queue.cc:1035] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:10.148442 10467 consensus_queue.cc:1035] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:10.166363 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:10.163043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:42911 (local address 127.7.171.1:35603)
0809 04:45:10.163167 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:10.163174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:10.163409 (+ 235us) server_negotiation.cc:407] Connection header received
0809 04:45:10.163567 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:10.163570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:10.163627 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:10.163697 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:10.163882 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.164396 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.165267 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.165548 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.165657 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:10.165897 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:10.165980 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:10.166056 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:10.166166 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 04:45:10.166209 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:45:10.166417 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:10.162957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:35603 (local address 127.7.171.3:42911)
0809 04:45:10.163326 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:10.163340 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:10.163457 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:10.163718 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:10.163720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:10.163727 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:10.163817 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:10.163821 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.164483 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.164485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:10.165140 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:10.165149 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:10.165664 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:10.165667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:10.165764 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:10.165802 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:10.166082 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:10.166088 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:45:10.166158 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 04:45:10.166246 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":97,"threads_started":1}
I20260809 04:45:10.190961 9773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:10.191126 9773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:10.193226 9773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3: Sending DeleteTablet for 3 replicas of tablet 6c0adeb23c51440ba4fd26732174effe
I20260809 04:45:10.193764 9932 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0adeb23c51440ba4fd26732174effe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:10 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:55820
I20260809 04:45:10.193861 10253 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0adeb23c51440ba4fd26732174effe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:10 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:59878
I20260809 04:45:10.193974 10090 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0adeb23c51440ba4fd26732174effe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:10 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:50154
I20260809 04:45:10.194061 10443 tablet_replica.cc:330] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8: stopping tablet replica
I20260809 04:45:10.194169 10444 tablet_replica.cc:330] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9: stopping tablet replica
I20260809 04:45:10.194180 10442 tablet_replica.cc:330] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6: stopping tablet replica
I20260809 04:45:10.194218 10443 raft_consensus.cc:2229] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:10.194288 10442 raft_consensus.cc:2229] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:10.194279 10444 raft_consensus.cc:2229] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:10.194336 10443 raft_consensus.cc:2258] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:10.194393 10442 raft_consensus.cc:2258] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:10.194602 10444 raft_consensus.cc:2258] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:10.195173 10442 ts_tablet_manager.cc:1905] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:10.195320 10444 ts_tablet_manager.cc:1905] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:10.196201 9773 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:10.196628 9773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:10.197875 10444 ts_tablet_manager.cc:1918] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:10.197876 10442 ts_tablet_manager.cc:1918] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:10.197957 10444 log.cc:1196] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-0/wal/wals/6c0adeb23c51440ba4fd26732174effe
I20260809 04:45:10.197957 10442 log.cc:1196] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/6c0adeb23c51440ba4fd26732174effe
I20260809 04:45:10.198235 10442 ts_tablet_manager.cc:1939] T 6c0adeb23c51440ba4fd26732174effe P 03a1d643cf8643f88ea0c27a799840b6: Deleting consensus metadata
I20260809 04:45:10.198235 10444 ts_tablet_manager.cc:1939] T 6c0adeb23c51440ba4fd26732174effe P 345d962f644249aab6f745b3a1b40af9: Deleting consensus metadata
I20260809 04:45:10.199065 10443 ts_tablet_manager.cc:1905] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:10.199173 9759 catalog_manager.cc:4841] TS 345d962f644249aab6f745b3a1b40af9 (127.7.171.1:35603): tablet 6c0adeb23c51440ba4fd26732174effe (table test-table [id=bd0003ce297a4a009edba7a4a303f142]) successfully deleted
I20260809 04:45:10.199906 9758 catalog_manager.cc:4841] TS 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2:36035): tablet 6c0adeb23c51440ba4fd26732174effe (table test-table [id=bd0003ce297a4a009edba7a4a303f142]) successfully deleted
I20260809 04:45:10.201673 10443 ts_tablet_manager.cc:1918] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:10.201758 10443 log.cc:1196] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/6c0adeb23c51440ba4fd26732174effe
I20260809 04:45:10.202016 10443 ts_tablet_manager.cc:1939] T 6c0adeb23c51440ba4fd26732174effe P ec60354b057d4b6695870b8562a136b8: Deleting consensus metadata
I20260809 04:45:10.202673 9759 catalog_manager.cc:4841] TS ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351): tablet 6c0adeb23c51440ba4fd26732174effe (table test-table [id=bd0003ce297a4a009edba7a4a303f142]) successfully deleted
I20260809 04:45:10.202802 9932 tablet_service.cc:1467] Processing CreateTablet for tablet 32a69e809d5944eb9ef971d9bb791248 (DEFAULT_TABLE table=test-table [id=86311ff799a14363b6c588e6bfb8a4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:10.203054 9932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a69e809d5944eb9ef971d9bb791248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:10.203305 10253 tablet_service.cc:1467] Processing CreateTablet for tablet 32a69e809d5944eb9ef971d9bb791248 (DEFAULT_TABLE table=test-table [id=86311ff799a14363b6c588e6bfb8a4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:10.203554 10253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a69e809d5944eb9ef971d9bb791248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:10.205209 10453 tablet_bootstrap.cc:492] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9: Bootstrap starting.
I20260809 04:45:10.205365 10090 tablet_service.cc:1467] Processing CreateTablet for tablet 32a69e809d5944eb9ef971d9bb791248 (DEFAULT_TABLE table=test-table [id=86311ff799a14363b6c588e6bfb8a4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:10.205607 10090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a69e809d5944eb9ef971d9bb791248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:10.206081 10453 tablet_bootstrap.cc:654] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:10.207170 10453 tablet_bootstrap.cc:492] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9: No bootstrap required, opened a new log
I20260809 04:45:10.207278 10453 ts_tablet_manager.cc:1397] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 04:45:10.207738 10455 tablet_bootstrap.cc:492] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6: Bootstrap starting.
I20260809 04:45:10.207721 10453 raft_consensus.cc:348] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.207794 10454 tablet_bootstrap.cc:492] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8: Bootstrap starting.
I20260809 04:45:10.207834 10453 raft_consensus.cc:374] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:10.207866 10453 raft_consensus.cc:729] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345d962f644249aab6f745b3a1b40af9, State: Initialized, Role: FOLLOWER
I20260809 04:45:10.207968 10453 consensus_queue.cc:260] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.208254 10453 ts_tablet_manager.cc:1428] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 04:45:10.208639 10455 tablet_bootstrap.cc:654] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:10.208648 10454 tablet_bootstrap.cc:654] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:10.210680 10455 tablet_bootstrap.cc:492] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6: No bootstrap required, opened a new log
I20260809 04:45:10.210772 10455 ts_tablet_manager.cc:1397] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:10.211175 10455 raft_consensus.cc:348] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.211289 10455 raft_consensus.cc:374] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:10.211313 10455 raft_consensus.cc:729] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a1d643cf8643f88ea0c27a799840b6, State: Initialized, Role: FOLLOWER
I20260809 04:45:10.211407 10455 consensus_queue.cc:260] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.211598 10455 ts_tablet_manager.cc:1428] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 04:45:10.212729 10454 tablet_bootstrap.cc:492] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8: No bootstrap required, opened a new log
I20260809 04:45:10.212823 10454 ts_tablet_manager.cc:1397] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 04:45:10.213217 10454 raft_consensus.cc:348] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.213330 10454 raft_consensus.cc:374] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:10.213369 10454 raft_consensus.cc:729] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec60354b057d4b6695870b8562a136b8, State: Initialized, Role: FOLLOWER
I20260809 04:45:10.213500 10454 consensus_queue.cc:260] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.213744 10454 ts_tablet_manager.cc:1428] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:10.242795 10467 raft_consensus.cc:482] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:10.243021 10467 raft_consensus.cc:504] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
W20260809 04:45:10.243294 10008 tablet.cc:2369] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:45:10.244207 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a69e809d5944eb9ef971d9bb791248" candidate_uuid: "345d962f644249aab6f745b3a1b40af9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a1d643cf8643f88ea0c27a799840b6" is_pre_election: true
I20260809 04:45:10.244453 10112 raft_consensus.cc:2454] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345d962f644249aab6f745b3a1b40af9 in term 0.
I20260809 04:45:10.244879 9879 leader_election.cc:302] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a1d643cf8643f88ea0c27a799840b6, 345d962f644249aab6f745b3a1b40af9; no voters:
I20260809 04:45:10.245199 10467 leader_election.cc:288] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2:36035), ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351)
I20260809 04:45:10.245307 10467 raft_consensus.cc:2781] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:10.245352 10467 raft_consensus.cc:482] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:10.245388 10467 raft_consensus.cc:3037] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:10.245906 10276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a69e809d5944eb9ef971d9bb791248" candidate_uuid: "345d962f644249aab6f745b3a1b40af9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60354b057d4b6695870b8562a136b8" is_pre_election: true
I20260809 04:45:10.246078 10276 raft_consensus.cc:2454] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345d962f644249aab6f745b3a1b40af9 in term 0.
I20260809 04:45:10.246783 10467 raft_consensus.cc:504] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.247195 10467 leader_election.cc:288] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [CANDIDATE]: Term 1 election: Requested vote from peers 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2:36035), ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351)
I20260809 04:45:10.247696 10276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a69e809d5944eb9ef971d9bb791248" candidate_uuid: "345d962f644249aab6f745b3a1b40af9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec60354b057d4b6695870b8562a136b8"
I20260809 04:45:10.247807 10276 raft_consensus.cc:3037] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:10.248366 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a69e809d5944eb9ef971d9bb791248" candidate_uuid: "345d962f644249aab6f745b3a1b40af9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a1d643cf8643f88ea0c27a799840b6"
I20260809 04:45:10.248502 10112 raft_consensus.cc:3037] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:10.248955 10276 raft_consensus.cc:2454] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 345d962f644249aab6f745b3a1b40af9 in term 1.
I20260809 04:45:10.249257 9880 leader_election.cc:302] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345d962f644249aab6f745b3a1b40af9, ec60354b057d4b6695870b8562a136b8; no voters:
I20260809 04:45:10.249408 10467 raft_consensus.cc:2781] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:10.249487 10467 raft_consensus.cc:686] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [term 1 LEADER]: Becoming Leader. State: Replica: 345d962f644249aab6f745b3a1b40af9, State: Running, Role: LEADER
I20260809 04:45:10.249609 10467 consensus_queue.cc:237] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } }
I20260809 04:45:10.251225 10112 raft_consensus.cc:2454] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 345d962f644249aab6f745b3a1b40af9 in term 1.
I20260809 04:45:10.251497 9773 catalog_manager.cc:5495] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 345d962f644249aab6f745b3a1b40af9 (127.7.171.1). New cstate: current_term: 1 leader_uuid: "345d962f644249aab6f745b3a1b40af9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345d962f644249aab6f745b3a1b40af9" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 35603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 } health_report { overall_health: UNKNOWN } } }
I20260809 04:45:10.277248 10276 raft_consensus.cc:1261] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Refusing update from remote peer 345d962f644249aab6f745b3a1b40af9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:10.277566 10112 raft_consensus.cc:1261] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Refusing update from remote peer 345d962f644249aab6f745b3a1b40af9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:10.277724 10467 consensus_queue.cc:1035] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec60354b057d4b6695870b8562a136b8" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:10.278266 10478 consensus_queue.cc:1035] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a1d643cf8643f88ea0c27a799840b6" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 04:45:10.293251 10420 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 04:45:11.847196 9773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:11.847354 9773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:11.849851 9773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f3e08f479194dd5af621c28ba6e8fa3: Sending DeleteTablet for 3 replicas of tablet 32a69e809d5944eb9ef971d9bb791248
I20260809 04:45:11.850457 9932 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a69e809d5944eb9ef971d9bb791248 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:11 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:55820
I20260809 04:45:11.850548 10253 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a69e809d5944eb9ef971d9bb791248 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:11 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:59878
I20260809 04:45:11.850600 10090 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a69e809d5944eb9ef971d9bb791248 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:11 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:50154
I20260809 04:45:11.850953 10516 tablet_replica.cc:330] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8: stopping tablet replica
I20260809 04:45:11.851007 10517 tablet_replica.cc:330] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6: stopping tablet replica
I20260809 04:45:11.851054 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 9860
I20260809 04:45:11.851119 10515 tablet_replica.cc:330] T 32a69e809d5944eb9ef971d9bb791248 P 345d962f644249aab6f745b3a1b40af9: stopping tablet replica
I20260809 04:45:11.855341 10516 raft_consensus.cc:2229] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:11.855561 10516 raft_consensus.cc:2258] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:11.856561 10516 ts_tablet_manager.cc:1905] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:11.851157 10517 raft_consensus.cc:2229] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:11.857245 10517 raft_consensus.cc:2258] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:11.857901 10517 ts_tablet_manager.cc:1905] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:11.859776 10516 ts_tablet_manager.cc:1918] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:45:11.859866 10516 log.cc:1196] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/32a69e809d5944eb9ef971d9bb791248
I20260809 04:45:11.860211 10516 ts_tablet_manager.cc:1939] T 32a69e809d5944eb9ef971d9bb791248 P ec60354b057d4b6695870b8562a136b8: Deleting consensus metadata
I20260809 04:45:11.860909 10517 ts_tablet_manager.cc:1918] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:45:11.860999 10517 log.cc:1196] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/32a69e809d5944eb9ef971d9bb791248
I20260809 04:45:11.861042 9759 catalog_manager.cc:4841] TS ec60354b057d4b6695870b8562a136b8 (127.7.171.3:45351): tablet 32a69e809d5944eb9ef971d9bb791248 (table test-table [id=86311ff799a14363b6c588e6bfb8a4d7]) successfully deleted
I20260809 04:45:11.861272 10517 ts_tablet_manager.cc:1939] T 32a69e809d5944eb9ef971d9bb791248 P 03a1d643cf8643f88ea0c27a799840b6: Deleting consensus metadata
I20260809 04:45:11.861922 9758 catalog_manager.cc:4841] TS 03a1d643cf8643f88ea0c27a799840b6 (127.7.171.2:36035): tablet 32a69e809d5944eb9ef971d9bb791248 (table test-table [id=86311ff799a14363b6c588e6bfb8a4d7]) successfully deleted
W20260809 04:45:11.864867 9759 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:35603 (error 108)
I20260809 04:45:11.865043 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 10016
I20260809 04:45:11.867851 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:11.867205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:35603 (local address 127.0.0.1:45860)
0809 04:45:11.867694 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:11.867762 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:35603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":335,"threads_started":1}
W20260809 04:45:11.867974 9759 catalog_manager.cc:4584] TS 345d962f644249aab6f745b3a1b40af9 (127.7.171.1:35603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 32a69e809d5944eb9ef971d9bb791248: Network error: Client connection negotiation failed: client connection to 127.7.171.1:35603: connect: Connection refused (error 111)
I20260809 04:45:11.875360 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 10180
I20260809 04:45:11.884676 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 9740
2026-08-09T04:45:11Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6130 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:11 dist-test-slave-1zqn krb5kdc[10522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:11 dist-test-slave-1zqn krb5kdc[10522](info): set up 2 sockets
Aug 09 04:45:11 dist-test-slave-1zqn krb5kdc[10522](info): commencing operation
krb5kdc: starting...
W20260809 04:45:13.027585 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:13 dist-test-slave-1zqn krb5kdc[10522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:13.159503 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:36169
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:40305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:36169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:13.275228 10537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:13.275664 10537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:13.275764 10537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:13.280325 10537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:13.280408 10537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:13.280431 10537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:13.280452 10537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:13.280471 10537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:13.285936 10537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:40305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:36169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:36169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:13.287070 10537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:13.288198 10537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:45:13.297309 10537 server_base.cc:1034] running on GCE node
W20260809 04:45:13.297271 10543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:13.297242 10545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:13.297287 10542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:13.298169 10537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:13.299348 10537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:13.300537 10537 hybrid_clock.cc:648] HybridClock initialized: now 1786250713300516 us; error 32 us; skew 500 ppm
Aug 09 04:45:13 dist-test-slave-1zqn krb5kdc[10522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250713, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:13.303505 10537 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:45:13.304545 10537 webserver.cc:460] Webserver started at http://127.7.171.62:34527/ using document root <none> and password file <none>
I20260809 04:45:13.305150 10537 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:13.305248 10537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:13.305498 10537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:13.307637 10537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "6a1647c6a0ff4a859f5da31419ca0ddf"
format_stamp: "Formatted at 2026-08-09 04:45:13 on dist-test-slave-1zqn"
server_key: "89d99321ca265c8cf91a11eab5602e2f"
server_key_iv: "beeee1e7bc9bcd9e5fae922f13e47d03"
server_key_version: "encryptionkey@0"
I20260809 04:45:13.308310 10537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "6a1647c6a0ff4a859f5da31419ca0ddf"
format_stamp: "Formatted at 2026-08-09 04:45:13 on dist-test-slave-1zqn"
server_key: "89d99321ca265c8cf91a11eab5602e2f"
server_key_iv: "beeee1e7bc9bcd9e5fae922f13e47d03"
server_key_version: "encryptionkey@0"
I20260809 04:45:13.312333 10537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:13.315011 10552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:13.316357 10537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:13.316520 10537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "6a1647c6a0ff4a859f5da31419ca0ddf"
format_stamp: "Formatted at 2026-08-09 04:45:13 on dist-test-slave-1zqn"
server_key: "89d99321ca265c8cf91a11eab5602e2f"
server_key_iv: "beeee1e7bc9bcd9e5fae922f13e47d03"
server_key_version: "encryptionkey@0"
I20260809 04:45:13.316653 10537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:13.347904 10537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:13.351130 10537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:13.351367 10537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:13.359642 10537 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:36169
I20260809 04:45:13.359658 10610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:36169 every 8 connection(s)
I20260809 04:45:13.360764 10537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:13.363585 10611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:13.366938 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 10537
I20260809 04:45:13.367064 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:13.367389 7852 external_mini_cluster.cc:1428] Setting key a3f3b90be00c76a6d3303bc09f4a0405
I20260809 04:45:13.369683 10611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf: Bootstrap starting.
I20260809 04:45:13.372767 10611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:13.373713 10611 log.cc:824] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:13.375697 10611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf: No bootstrap required, opened a new log
Aug 09 04:45:13 dist-test-slave-1zqn krb5kdc[10522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:13.379739 10611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1647c6a0ff4a859f5da31419ca0ddf" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36169 } }
I20260809 04:45:13.380000 10611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:13.380098 10611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a1647c6a0ff4a859f5da31419ca0ddf, State: Initialized, Role: FOLLOWER
I20260809 04:45:13.380599 10611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1647c6a0ff4a859f5da31419ca0ddf" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36169 } }
I20260809 04:45:13.380753 10611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:13.380836 10611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:13.380942 10611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:13.381950 10611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1647c6a0ff4a859f5da31419ca0ddf" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36169 } }
I20260809 04:45:13.382306 10611 leader_election.cc:302] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a1647c6a0ff4a859f5da31419ca0ddf; no voters:
I20260809 04:45:13.382623 10611 leader_election.cc:288] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:13.382735 10617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:13.382896 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:13.368942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44458 (local address 127.7.171.62:36169)
0809 04:45:13.369371 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:13.369395 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:13.369432 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:45:13.370222 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:13.370243 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:13.370592 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:13.370888 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:13.371874 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:13.373035 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:13.374227 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:13.374708 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:13.374884 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:13.377484 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:13.377489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:13.377502 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:13.377530 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:13.379479 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:13.379983 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:13.379988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:13.379993 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:13.380246 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:13.380536 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:13.380540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:13.380542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:13.381077 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:13.381304 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:13.381601 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 04:45:13.381907 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":145,"threads_started":1}
I20260809 04:45:13.382956 10617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [term 1 LEADER]: Becoming Leader. State: Replica: 6a1647c6a0ff4a859f5da31419ca0ddf, State: Running, Role: LEADER
I20260809 04:45:13.383255 10617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1647c6a0ff4a859f5da31419ca0ddf" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36169 } }
I20260809 04:45:13.383989 10611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:13.384914 10618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a1647c6a0ff4a859f5da31419ca0ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1647c6a0ff4a859f5da31419ca0ddf" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36169 } } }
I20260809 04:45:13.385023 10618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:13.385394 10619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a1647c6a0ff4a859f5da31419ca0ddf. Latest consensus state: current_term: 1 leader_uuid: "6a1647c6a0ff4a859f5da31419ca0ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a1647c6a0ff4a859f5da31419ca0ddf" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36169 } } }
I20260809 04:45:13.385485 10619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:13.386454 10626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:13.389078 10626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:13.394374 10626 catalog_manager.cc:1347] Generated new cluster ID: 55b2993a3ba9439a9ecbce2cde1fbb1a
I20260809 04:45:13.394483 10626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:13.405377 10626 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:13.406668 10626 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:13.416834 10626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a1647c6a0ff4a859f5da31419ca0ddf: Generated new TSK 0
I20260809 04:45:13.417666 10626 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:45:13.459278 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36169
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:40305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:13.576260 10639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:13.576586 10639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:13.576666 10639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:13.580786 10639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:13.580904 10639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:13.580999 10639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:13.586727 10639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:40305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:13.587862 10639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:13.588956 10639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:13.599049 10646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:13.599073 10649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:13.599049 10647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:13.599115 10639 server_base.cc:1034] running on GCE node
I20260809 04:45:13.599785 10639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:13.600517 10639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:13.601698 10639 hybrid_clock.cc:648] HybridClock initialized: now 1786250713601675 us; error 36 us; skew 500 ppm
Aug 09 04:45:13 dist-test-slave-1zqn krb5kdc[10522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250713, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:13.604688 10639 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:45:13.605566 10639 webserver.cc:460] Webserver started at http://127.7.171.1:34009/ using document root <none> and password file <none>
I20260809 04:45:13.606132 10639 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:13.606194 10639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:13.606410 10639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:13.608601 10639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "20e2c85961bf4cd9ad867f1bda46e4ee"
format_stamp: "Formatted at 2026-08-09 04:45:13 on dist-test-slave-1zqn"
server_key: "f854d9e2bc953ceda4bd40e4adaff27c"
server_key_iv: "5ce51dae4f72107a5e7b606932a94e8b"
server_key_version: "encryptionkey@0"
I20260809 04:45:13.609185 10639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "20e2c85961bf4cd9ad867f1bda46e4ee"
format_stamp: "Formatted at 2026-08-09 04:45:13 on dist-test-slave-1zqn"
server_key: "f854d9e2bc953ceda4bd40e4adaff27c"
server_key_iv: "5ce51dae4f72107a5e7b606932a94e8b"
server_key_version: "encryptionkey@0"
I20260809 04:45:13.613078 10639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:13.615513 10656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:13.616724 10639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:13.616854 10639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "20e2c85961bf4cd9ad867f1bda46e4ee"
format_stamp: "Formatted at 2026-08-09 04:45:13 on dist-test-slave-1zqn"
server_key: "f854d9e2bc953ceda4bd40e4adaff27c"
server_key_iv: "5ce51dae4f72107a5e7b606932a94e8b"
server_key_version: "encryptionkey@0"
I20260809 04:45:13.616962 10639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:13.635418 10639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:13.638839 10639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:13.639053 10639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:13.639660 10639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:13.640588 10639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:13.640655 10639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:13.640727 10639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:13.640767 10639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:13.649721 10639 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:42477
I20260809 04:45:13.649758 10783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:42477 every 8 connection(s)
I20260809 04:45:13.650655 10639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:13.657241 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 10639
I20260809 04:45:13.657362 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:13.657658 7852 external_mini_cluster.cc:1428] Setting key d27ef3c896bf16c78e976ace8785d856
Aug 09 04:45:13 dist-test-slave-1zqn krb5kdc[10522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250713, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:13.665460 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:13.652658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:48225 (local address 127.7.171.62:36169)
0809 04:45:13.652809 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:13.652813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:13.653868 (+ 1055us) server_negotiation.cc:407] Connection header received
0809 04:45:13.654774 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:13.654777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:13.654835 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:13.654917 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:13.656112 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:13.656652 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:13.658214 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:13.658455 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:13.658565 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:13.661239 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:13.661243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:13.661245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:13.661267 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:13.662982 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:13.663631 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:13.663635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:13.663636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:13.663748 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:13.664198 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:13.664201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:13.664202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:13.664418 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:13.664517 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:13.665219 (+ 702us) server_negotiation.cc:299] Negotiation successful
0809 04:45:13.665334 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:45:13.666713 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:13.653028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36169 (local address 127.7.171.1:48225)
0809 04:45:13.653704 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:13.653748 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:13.654559 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:13.655089 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:13.655098 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:13.655457 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:13.655930 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:13.655943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:13.656779 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:13.656788 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:13.658092 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:13.658102 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:13.658613 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:13.658618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:13.658823 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:13.659435 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:13.659451 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:13.661068 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:13.663107 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:13.663115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:13.663134 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:13.663506 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:13.663857 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:13.663863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:13.663868 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:13.664097 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:13.664547 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:13.664557 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:13.664906 (+ 349us) client_negotiation.cc:769] Sending connection context
0809 04:45:13.665159 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 04:45:13.665528 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":114,"threads_started":1}
I20260809 04:45:13.668017 10786 heartbeater.cc:347] Connected to a master server at 127.7.171.62:36169
I20260809 04:45:13.668314 10786 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:13.668939 10786 heartbeater.cc:511] Master 127.7.171.62:36169 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:45:13.670584 10571 ts_manager.cc:194] Registered new tserver with Master: 20e2c85961bf4cd9ad867f1bda46e4ee (127.7.171.1:42477)
I20260809 04:45:13.671819 10571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:48225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:45:13.691145 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36169
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:40305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:13.806315 10791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:13.806574 10791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:13.806617 10791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:13.810614 10791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:13.810681 10791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:13.810786 10791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:13.816181 10791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:40305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:13.817168 10791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:13.818210 10791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:45:13.828778 10791 server_base.cc:1034] running on GCE node
W20260809 04:45:13.828778 10796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:13.828781 10797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:13.828826 10799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:13.829447 10791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:13.830094 10791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:13.831314 10791 hybrid_clock.cc:648] HybridClock initialized: now 1786250713831284 us; error 47 us; skew 500 ppm
Aug 09 04:45:13 dist-test-slave-1zqn krb5kdc[10522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250713, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:13.834381 10791 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:45:13.835389 10791 webserver.cc:460] Webserver started at http://127.7.171.2:35135/ using document root <none> and password file <none>
I20260809 04:45:13.836020 10791 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:13.836155 10791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:13.836361 10791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:13.838605 10791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "8a55b6c5b7b945cfb73c7835f9ee9845"
format_stamp: "Formatted at 2026-08-09 04:45:13 on dist-test-slave-1zqn"
server_key: "d2c416d9afe07d86fc4b3a9dd7f0bfee"
server_key_iv: "4c3758c676ed10e5944977b410499f71"
server_key_version: "encryptionkey@0"
I20260809 04:45:13.839184 10791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "8a55b6c5b7b945cfb73c7835f9ee9845"
format_stamp: "Formatted at 2026-08-09 04:45:13 on dist-test-slave-1zqn"
server_key: "d2c416d9afe07d86fc4b3a9dd7f0bfee"
server_key_iv: "4c3758c676ed10e5944977b410499f71"
server_key_version: "encryptionkey@0"
I20260809 04:45:13.843627 10791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 04:45:13.846381 10807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:13.848189 10791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:13.848349 10791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "8a55b6c5b7b945cfb73c7835f9ee9845"
format_stamp: "Formatted at 2026-08-09 04:45:13 on dist-test-slave-1zqn"
server_key: "d2c416d9afe07d86fc4b3a9dd7f0bfee"
server_key_iv: "4c3758c676ed10e5944977b410499f71"
server_key_version: "encryptionkey@0"
I20260809 04:45:13.848502 10791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:13.878542 10791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:13.882159 10791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:13.882373 10791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:13.883001 10791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:13.883867 10791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:13.883937 10791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:13.884006 10791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:13.884052 10791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:13.893496 10791 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:44517
I20260809 04:45:13.893530 10932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:44517 every 8 connection(s)
I20260809 04:45:13.894498 10791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:13.898998 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 10791
I20260809 04:45:13.899150 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:13.899416 7852 external_mini_cluster.cc:1428] Setting key f8ee3cf385ca57acd66110b7fdda95c4
Aug 09 04:45:13 dist-test-slave-1zqn krb5kdc[10522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250713, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:13.909694 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:13.896545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:40325 (local address 127.7.171.62:36169)
0809 04:45:13.896705 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:13.896711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:13.897809 (+ 1098us) server_negotiation.cc:407] Connection header received
0809 04:45:13.898707 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:13.898713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:13.898779 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:13.898887 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:13.900276 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:13.901146 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:13.902902 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:13.903391 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:13.903529 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:13.906002 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:13.906006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:13.906008 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:13.906033 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:13.907546 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:13.908096 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:13.908104 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:13.908106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:13.908237 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:13.908582 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:13.908587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:13.908588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:13.908780 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:13.908886 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:13.909393 (+ 507us) server_negotiation.cc:299] Negotiation successful
0809 04:45:13.909526 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:45:13.910655 10936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:13.896979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36169 (local address 127.7.171.2:40325)
0809 04:45:13.897665 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:13.897709 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:13.898463 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:13.899037 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:13.899045 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:13.899469 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:13.900107 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:13.900130 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:13.901285 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:13.901288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:13.902748 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:13.902764 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:13.903549 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:13.903552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:13.903702 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:13.904356 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:13.904373 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:13.905828 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:13.907674 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:13.907680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:13.907695 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:13.907965 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:13.908332 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:13.908335 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:13.908337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:13.908487 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:13.908886 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:13.908894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:13.909156 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 04:45:13.909349 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 04:45:13.909704 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":129,"threads_started":1}
I20260809 04:45:13.911870 10933 heartbeater.cc:347] Connected to a master server at 127.7.171.62:36169
I20260809 04:45:13.912179 10933 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:13.912737 10933 heartbeater.cc:511] Master 127.7.171.62:36169 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
I20260809 04:45:13.914009 10571 ts_manager.cc:194] Registered new tserver with Master: 8a55b6c5b7b945cfb73c7835f9ee9845 (127.7.171.2:44517)
I20260809 04:45:13.914877 10571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:40325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:45:13.931586 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36169
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:40305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:14.050379 10939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:14.050639 10939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:14.050681 10939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:14.054639 10939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:14.054694 10939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:14.054750 10939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:14.059922 10939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:40305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:14.060930 10939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:14.062155 10939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:14.071879 10948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:14.071869 10944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:14.072119 10939 server_base.cc:1034] running on GCE node
W20260809 04:45:14.071869 10946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:14.072626 10939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:14.073273 10939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:14.074467 10939 hybrid_clock.cc:648] HybridClock initialized: now 1786250714074444 us; error 43 us; skew 500 ppm
Aug 09 04:45:14 dist-test-slave-1zqn krb5kdc[10522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250714, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:14.077308 10939 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:45:14.078205 10939 webserver.cc:460] Webserver started at http://127.7.171.3:42759/ using document root <none> and password file <none>
I20260809 04:45:14.078809 10939 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:14.078902 10939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:14.079141 10939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:14.081296 10939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "de7f8307b69e4b4d97aca2197ea24d6b"
format_stamp: "Formatted at 2026-08-09 04:45:14 on dist-test-slave-1zqn"
server_key: "1c712c37e1dae7a080634dfc227fa02c"
server_key_iv: "0a1ad678147e2a0759dc6c97f56917d0"
server_key_version: "encryptionkey@0"
I20260809 04:45:14.081862 10939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "de7f8307b69e4b4d97aca2197ea24d6b"
format_stamp: "Formatted at 2026-08-09 04:45:14 on dist-test-slave-1zqn"
server_key: "1c712c37e1dae7a080634dfc227fa02c"
server_key_iv: "0a1ad678147e2a0759dc6c97f56917d0"
server_key_version: "encryptionkey@0"
I20260809 04:45:14.085832 10939 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:45:14.088340 10955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:14.089749 10939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:14.089888 10939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "de7f8307b69e4b4d97aca2197ea24d6b"
format_stamp: "Formatted at 2026-08-09 04:45:14 on dist-test-slave-1zqn"
server_key: "1c712c37e1dae7a080634dfc227fa02c"
server_key_iv: "0a1ad678147e2a0759dc6c97f56917d0"
server_key_version: "encryptionkey@0"
I20260809 04:45:14.090021 10939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:14.103688 10939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:14.107075 10939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:14.107280 10939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:14.107888 10939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:14.108783 10939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:14.108861 10939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:14.108928 10939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:14.108978 10939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:14.117913 10939 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:38593
I20260809 04:45:14.117969 11080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:38593 every 8 connection(s)
I20260809 04:45:14.119091 10939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:14.119360 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 10939
I20260809 04:45:14.119555 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:14.119905 7852 external_mini_cluster.cc:1428] Setting key 365b061dcbf0cd8aaa4967d608558a06
Aug 09 04:45:14 dist-test-slave-1zqn krb5kdc[10522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250714, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:14.133394 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:14.122978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:59559 (local address 127.7.171.62:36169)
0809 04:45:14.123116 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:14.123120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:14.123132 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:14.123765 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:14.123767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:14.123807 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:14.123867 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:14.125204 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:14.125720 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:14.126967 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:14.127179 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:14.127257 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:14.129672 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:14.129675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:14.129677 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:14.129702 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:14.131187 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:14.131637 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:14.131639 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:14.131640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:14.131712 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:14.132174 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:14.132177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:14.132179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:14.132382 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:14.132475 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:14.133074 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 04:45:14.133185 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 04:45:14.134466 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:14.121867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36169 (local address 127.7.171.3:59559)
0809 04:45:14.122659 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:14.122715 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:14.123574 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:14.124008 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:14.124015 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:14.124550 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:14.125072 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:14.125086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:14.125827 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:14.125829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:14.126838 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:14.126848 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:14.127330 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:14.127334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:14.127481 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:14.128010 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:14.128026 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:14.129524 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:14.131292 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:14.131297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:14.131308 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:14.131554 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:14.131787 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:14.131789 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:14.131791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:14.131984 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:14.132489 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:14.132495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:14.132744 (+ 249us) client_negotiation.cc:769] Sending connection context
0809 04:45:14.133014 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 04:45:14.133384 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":153,"threads_started":1}
I20260809 04:45:14.135584 11081 heartbeater.cc:347] Connected to a master server at 127.7.171.62:36169
I20260809 04:45:14.135830 11081 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:14.136451 11081 heartbeater.cc:511] Master 127.7.171.62:36169 requested a full tablet report, sending...
I20260809 04:45:14.137497 10571 ts_manager.cc:194] Registered new tserver with Master: de7f8307b69e4b4d97aca2197ea24d6b (127.7.171.3:38593)
I20260809 04:45:14.138087 10571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:59559
I20260809 04:45:14.146317 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:45:14.161620 10614 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:14.159939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44460 (local address 127.7.171.62:36169)
0809 04:45:14.160148 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:14.160154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:14.160347 (+ 193us) server_negotiation.cc:407] Connection header received
0809 04:45:14.160480 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:14.160483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:14.160531 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:14.160612 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:14.161407 (+ 795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44460: BlockingRecv error: recv got EOF from 127.0.0.1:44460 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:45:14.162679 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 10639
I20260809 04:45:14.173359 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 10791
I20260809 04:45:14.183517 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 10939
I20260809 04:45:14.192404 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 10537
2026-08-09T04:45:14Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2307 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:14 dist-test-slave-1zqn krb5kdc[11094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:14 dist-test-slave-1zqn krb5kdc[11094](info): set up 2 sockets
Aug 09 04:45:14 dist-test-slave-1zqn krb5kdc[11094](info): commencing operation
krb5kdc: starting...
W20260809 04:45:15.314503 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:15 dist-test-slave-1zqn krb5kdc[11094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:15.447548 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46075
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:46075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:15.561481 11110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:15.561769 11110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:15.561829 11110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:15.566045 11110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:15.566111 11110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:15.566131 11110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:15.566181 11110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:15.566216 11110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:15.571543 11110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:46075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:15.572649 11110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:15.573691 11110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:15.582808 11116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:15.582887 11110 server_base.cc:1034] running on GCE node
W20260809 04:45:15.582803 11119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:15.582801 11117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:15.583856 11110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:15.585156 11110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:15.586354 11110 hybrid_clock.cc:648] HybridClock initialized: now 1786250715586342 us; error 29 us; skew 500 ppm
Aug 09 04:45:15 dist-test-slave-1zqn krb5kdc[11094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250715, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:15.596439 11110 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:45:15.597548 11110 webserver.cc:460] Webserver started at http://127.7.171.62:36411/ using document root <none> and password file <none>
I20260809 04:45:15.598186 11110 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:15.598258 11110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:15.598523 11110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:15.601495 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "3f89682dda3a489c8b352f4800007641"
format_stamp: "Formatted at 2026-08-09 04:45:15 on dist-test-slave-1zqn"
server_key: "0498e4f6749221ee04221e8e3b18bdf8"
server_key_iv: "958c56d1044fa8d005c3f62b9e7dc41b"
server_key_version: "encryptionkey@0"
I20260809 04:45:15.602253 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "3f89682dda3a489c8b352f4800007641"
format_stamp: "Formatted at 2026-08-09 04:45:15 on dist-test-slave-1zqn"
server_key: "0498e4f6749221ee04221e8e3b18bdf8"
server_key_iv: "958c56d1044fa8d005c3f62b9e7dc41b"
server_key_version: "encryptionkey@0"
I20260809 04:45:15.606670 11110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:45:15.609478 11127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:15.610782 11110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:45:15.610944 11110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "3f89682dda3a489c8b352f4800007641"
format_stamp: "Formatted at 2026-08-09 04:45:15 on dist-test-slave-1zqn"
server_key: "0498e4f6749221ee04221e8e3b18bdf8"
server_key_iv: "958c56d1044fa8d005c3f62b9e7dc41b"
server_key_version: "encryptionkey@0"
I20260809 04:45:15.611074 11110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:15.624577 11110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:15.628263 11110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:15.628489 11110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:15.635883 11110 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:46075
I20260809 04:45:15.636211 11189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:46075 every 8 connection(s)
I20260809 04:45:15.636983 11110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:15.639817 11190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:15.645362 11190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641: Bootstrap starting.
I20260809 04:45:15.645766 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 11110
I20260809 04:45:15.645927 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:15.646219 7852 external_mini_cluster.cc:1428] Setting key 2eb2cedc5eb80bc42e0834a4113297d2
I20260809 04:45:15.648119 11190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:15.648906 11190 log.cc:824] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:15.650830 11190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641: No bootstrap required, opened a new log
I20260809 04:45:15.654316 11190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f89682dda3a489c8b352f4800007641" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46075 } }
I20260809 04:45:15.654557 11190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:15.654647 11190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f89682dda3a489c8b352f4800007641, State: Initialized, Role: FOLLOWER
Aug 09 04:45:15 dist-test-slave-1zqn krb5kdc[11094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:15.655126 11190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f89682dda3a489c8b352f4800007641" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46075 } }
I20260809 04:45:15.655315 11190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:15.655402 11190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:15.655498 11190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:15.656654 11190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f89682dda3a489c8b352f4800007641" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46075 } }
I20260809 04:45:15.657042 11190 leader_election.cc:302] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f89682dda3a489c8b352f4800007641; no voters:
I20260809 04:45:15.657384 11190 leader_election.cc:288] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:15.657526 11195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:15.657768 11195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [term 1 LEADER]: Becoming Leader. State: Replica: 3f89682dda3a489c8b352f4800007641, State: Running, Role: LEADER
I20260809 04:45:15.658111 11195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f89682dda3a489c8b352f4800007641" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46075 } }
I20260809 04:45:15.658876 11190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:15.660462 11196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f89682dda3a489c8b352f4800007641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f89682dda3a489c8b352f4800007641" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46075 } } }
I20260809 04:45:15.660596 11196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:15.660986 11207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:15.661034 11197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f89682dda3a489c8b352f4800007641. Latest consensus state: current_term: 1 leader_uuid: "3f89682dda3a489c8b352f4800007641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f89682dda3a489c8b352f4800007641" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46075 } } }
I20260809 04:45:15.661950 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:15.647510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53418 (local address 127.7.171.62:46075)
0809 04:45:15.647965 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:15.647982 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:15.648018 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:45:15.648837 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:15.648856 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:15.649277 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:15.649625 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:15.650507 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:15.651552 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:15.652511 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:15.652995 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:15.653189 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:15.655846 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:15.655851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:15.655865 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:15.655892 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:15.657966 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:15.658476 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:15.658482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:15.658488 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:15.658648 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:15.658980 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:15.658983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:15.658985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:15.659514 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:15.659730 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:15.660581 (+ 851us) server_negotiation.cc:299] Negotiation successful
0809 04:45:15.660918 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":153,"threads_started":1}
I20260809 04:45:15.662220 11197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:15.664470 11207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:15.665668 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:45:15.670832 11207 catalog_manager.cc:1347] Generated new cluster ID: 131c2f0e5daa441f9d4a97ee78ef9c15
I20260809 04:45:15.670905 11207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:15.679008 11207 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:15.679970 11207 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:15.687894 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:15.682174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53434 (local address 127.7.171.62:46075)
0809 04:45:15.682284 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:15.682289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:15.682562 (+ 273us) server_negotiation.cc:407] Connection header received
0809 04:45:15.682701 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:15.682705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:15.682751 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:15.682846 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:15.683457 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:15.684270 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:15.685497 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:15.685835 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:15.685920 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:15.686851 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:15.686855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:15.686858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:15.687430 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:15.687462 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:15.687480 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:15.687562 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:15.687692 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 04:45:15.687755 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 04:45:15.693646 11207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f89682dda3a489c8b352f4800007641: Generated new TSK 0
I20260809 04:45:15.694435 11207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:45:15.702318 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 11110
2026-08-09T04:45:15Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1507 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:15 dist-test-slave-1zqn krb5kdc[11226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:15 dist-test-slave-1zqn krb5kdc[11226](info): set up 2 sockets
Aug 09 04:45:15 dist-test-slave-1zqn krb5kdc[11226](info): commencing operation
krb5kdc: starting...
W20260809 04:45:16.843287 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:16 dist-test-slave-1zqn krb5kdc[11226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:16.975221 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34801
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:34801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:17.093233 11241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:17.093585 11241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:17.093668 11241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:17.097858 11241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:17.097970 11241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:17.098016 11241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:17.098080 11241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:17.098124 11241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:17.103691 11241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:34801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:17.104854 11241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:17.105928 11241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:17.115160 11249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:17.115191 11250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:17.115466 11241 server_base.cc:1034] running on GCE node
W20260809 04:45:17.115160 11252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:17.116272 11241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:17.117400 11241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:17.118611 11241 hybrid_clock.cc:648] HybridClock initialized: now 1786250717118592 us; error 36 us; skew 500 ppm
Aug 09 04:45:17 dist-test-slave-1zqn krb5kdc[11226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:17.121729 11241 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:45:17.122707 11241 webserver.cc:460] Webserver started at http://127.7.171.62:35919/ using document root <none> and password file <none>
I20260809 04:45:17.123286 11241 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:17.123392 11241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:17.123642 11241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:17.125842 11241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "c7dc6eee7062498a942bac6193788ed0"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "aab7193734d835bbdba181f53fe0b74e"
server_key_iv: "c246ee7a43e94e0d7b079f819400c71b"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.126462 11241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "c7dc6eee7062498a942bac6193788ed0"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "aab7193734d835bbdba181f53fe0b74e"
server_key_iv: "c246ee7a43e94e0d7b079f819400c71b"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.130421 11241 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 04:45:17.132982 11259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:17.134305 11241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:17.134475 11241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "c7dc6eee7062498a942bac6193788ed0"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "aab7193734d835bbdba181f53fe0b74e"
server_key_iv: "c246ee7a43e94e0d7b079f819400c71b"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.134604 11241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:17.165311 11241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:17.169018 11241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:17.169258 11241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:17.177026 11241 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:34801
I20260809 04:45:17.177363 11322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:34801 every 8 connection(s)
I20260809 04:45:17.178097 11241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:17.181067 11323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:17.182870 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 11241
I20260809 04:45:17.183110 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:17.183426 7852 external_mini_cluster.cc:1428] Setting key 809d331d1ef21f91f18babdf15ca9d64
I20260809 04:45:17.187286 11323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0: Bootstrap starting.
I20260809 04:45:17.189936 11323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:17.190800 11323 log.cc:824] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:17.192988 11323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0: No bootstrap required, opened a new log
Aug 09 04:45:17 dist-test-slave-1zqn krb5kdc[11226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:17.198462 11323 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dc6eee7062498a942bac6193788ed0" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34801 } }
I20260809 04:45:17.198796 11323 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:17.198858 11323 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7dc6eee7062498a942bac6193788ed0, State: Initialized, Role: FOLLOWER
I20260809 04:45:17.199383 11323 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dc6eee7062498a942bac6193788ed0" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34801 } }
I20260809 04:45:17.199559 11323 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:17.199622 11323 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:17.199724 11323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:17.199836 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:17.184950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.7.171.62:34801)
0809 04:45:17.185408 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:17.185430 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:17.185469 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:45:17.186398 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:17.186435 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:17.186806 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:17.187115 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:17.188166 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.189294 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.190507 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.191118 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.191314 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:17.193960 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:17.193967 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:17.193986 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:17.194022 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:17.196302 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:17.196746 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:17.196752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:17.196759 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:17.196927 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:17.197229 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:17.197233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:17.197235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:17.197829 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:17.198061 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:17.198356 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 04:45:17.198710 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":146,"threads_started":1}
I20260809 04:45:17.200871 11323 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dc6eee7062498a942bac6193788ed0" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34801 } }
I20260809 04:45:17.201268 11323 leader_election.cc:302] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7dc6eee7062498a942bac6193788ed0; no voters:
I20260809 04:45:17.201642 11323 leader_election.cc:288] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:17.201783 11328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:17.202018 11328 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [term 1 LEADER]: Becoming Leader. State: Replica: c7dc6eee7062498a942bac6193788ed0, State: Running, Role: LEADER
I20260809 04:45:17.202366 11328 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dc6eee7062498a942bac6193788ed0" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34801 } }
I20260809 04:45:17.203056 11323 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:17.204255 11329 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7dc6eee7062498a942bac6193788ed0. Latest consensus state: current_term: 1 leader_uuid: "c7dc6eee7062498a942bac6193788ed0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dc6eee7062498a942bac6193788ed0" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34801 } } }
I20260809 04:45:17.204232 11330 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7dc6eee7062498a942bac6193788ed0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7dc6eee7062498a942bac6193788ed0" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34801 } } }
I20260809 04:45:17.204399 11330 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:17.204399 11329 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:17.205040 11337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:17.208189 11337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:17.215778 11337 catalog_manager.cc:1347] Generated new cluster ID: 5279fa54290a4b43b4cfeb3c027640e5
I20260809 04:45:17.215890 11337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:17.242424 11337 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:17.243844 11337 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:17.253768 11337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7dc6eee7062498a942bac6193788ed0: Generated new TSK 0
I20260809 04:45:17.254706 11337 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:45:17.287880 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34801
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:17.406445 11352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:17.406715 11352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:17.406759 11352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:17.410715 11352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:17.410769 11352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:17.410825 11352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:17.416144 11352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:17.417181 11352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:17.418160 11352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:17.428467 11357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:17.428458 11360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:17.428465 11358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:17.428817 11352 server_base.cc:1034] running on GCE node
I20260809 04:45:17.429302 11352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:17.429962 11352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:17.431180 11352 hybrid_clock.cc:648] HybridClock initialized: now 1786250717431151 us; error 32 us; skew 500 ppm
Aug 09 04:45:17 dist-test-slave-1zqn krb5kdc[11226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:17.434268 11352 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:45:17.435334 11352 webserver.cc:460] Webserver started at http://127.7.171.1:42337/ using document root <none> and password file <none>
I20260809 04:45:17.435953 11352 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:17.436054 11352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:17.436332 11352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:17.438649 11352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "8c9c5c9c2a8442f3a36654743a135b51"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "94d81008c178a2c5649e6e69c36a630c"
server_key_iv: "1af49c2c8932f0c6d82cdfa7c23a92ab"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.439298 11352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "8c9c5c9c2a8442f3a36654743a135b51"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "94d81008c178a2c5649e6e69c36a630c"
server_key_iv: "1af49c2c8932f0c6d82cdfa7c23a92ab"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.443425 11352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260809 04:45:17.446341 11367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:17.447728 11352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:17.447896 11352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "8c9c5c9c2a8442f3a36654743a135b51"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "94d81008c178a2c5649e6e69c36a630c"
server_key_iv: "1af49c2c8932f0c6d82cdfa7c23a92ab"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.448030 11352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:17.469489 11352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:17.472899 11352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:17.473133 11352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:17.473810 11352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:17.474761 11352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:17.474839 11352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:17.474920 11352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:17.474977 11352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:17.484885 11352 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:42101
I20260809 04:45:17.484915 11494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:42101 every 8 connection(s)
I20260809 04:45:17.485983 11352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
Aug 09 04:45:17 dist-test-slave-1zqn krb5kdc[11226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:17.496327 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 11352
I20260809 04:45:17.496466 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:17.496784 7852 external_mini_cluster.cc:1428] Setting key bef23a22eb5288ef4eb44443e9404926
I20260809 04:45:17.500905 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:17.488028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:34625 (local address 127.7.171.62:34801)
0809 04:45:17.488219 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:17.488223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:17.489197 (+ 974us) server_negotiation.cc:407] Connection header received
0809 04:45:17.490579 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:17.490582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:17.490650 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:17.490738 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:17.491942 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.492626 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.493755 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.494014 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.494106 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:17.496865 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:17.496869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:17.496873 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:17.496904 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:17.498566 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:17.499084 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:17.499090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:17.499095 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:17.499190 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:17.499525 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:17.499531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:17.499536 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:17.499796 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:17.499904 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:17.500526 (+ 622us) server_negotiation.cc:299] Negotiation successful
0809 04:45:17.500680 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260809 04:45:17.501996 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:17.488537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34801 (local address 127.7.171.1:34625)
0809 04:45:17.489017 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:17.489063 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:17.490339 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:17.490906 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:17.490913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:17.491279 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:17.491790 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:17.491804 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.492755 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.492757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:17.493636 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:17.493645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.494156 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.494158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:17.494302 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:17.494966 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:17.494982 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:17.496681 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:17.498692 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:17.498698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:17.498711 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:17.498975 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:17.499286 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:17.499289 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:17.499290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:17.499430 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:17.499901 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:17.499906 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:17.500259 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 04:45:17.500440 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 04:45:17.500811 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":121,"threads_started":1}
I20260809 04:45:17.503619 11495 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34801
I20260809 04:45:17.503937 11495 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:17.504599 11495 heartbeater.cc:511] Master 127.7.171.62:34801 requested a full tablet report, sending...
I20260809 04:45:17.506570 11280 ts_manager.cc:194] Registered new tserver with Master: 8c9c5c9c2a8442f3a36654743a135b51 (127.7.171.1:42101)
I20260809 04:45:17.508402 11280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:34625
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:45:17.527567 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34801
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:17.650295 11500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:17.650579 11500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:17.650624 11500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:17.654826 11500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:17.654891 11500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:17.654950 11500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:17.660202 11500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:17.661252 11500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:17.662213 11500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:17.672590 11509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:17.672657 11506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:17.672550 11507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:17.672847 11500 server_base.cc:1034] running on GCE node
I20260809 04:45:17.673292 11500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:17.674000 11500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:17.675184 11500 hybrid_clock.cc:648] HybridClock initialized: now 1786250717675167 us; error 29 us; skew 500 ppm
Aug 09 04:45:17 dist-test-slave-1zqn krb5kdc[11226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:17.678196 11500 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:45:17.679229 11500 webserver.cc:460] Webserver started at http://127.7.171.2:46689/ using document root <none> and password file <none>
I20260809 04:45:17.679854 11500 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:17.679955 11500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:17.680258 11500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:17.682539 11500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "b8fcb978fd9e437a911ef25fe21c8a68"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "a8c1ec46d95d050c0a4e3734370d36dd"
server_key_iv: "b83233e467ce25dad969a2540d7e7a89"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.683182 11500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "b8fcb978fd9e437a911ef25fe21c8a68"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "a8c1ec46d95d050c0a4e3734370d36dd"
server_key_iv: "b83233e467ce25dad969a2540d7e7a89"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.687652 11500 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260809 04:45:17.690385 11516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:17.691756 11500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:17.691901 11500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "b8fcb978fd9e437a911ef25fe21c8a68"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "a8c1ec46d95d050c0a4e3734370d36dd"
server_key_iv: "b83233e467ce25dad969a2540d7e7a89"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.691998 11500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:17.710866 11500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:17.714274 11500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:17.714473 11500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:17.715062 11500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:17.715970 11500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:17.716027 11500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:17.716128 11500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:17.716156 11500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:17.725476 11500 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:39023
I20260809 04:45:17.725508 11648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:39023 every 8 connection(s)
I20260809 04:45:17.726442 11500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:17.736104 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 11500
I20260809 04:45:17.736249 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:17.736567 7852 external_mini_cluster.cc:1428] Setting key 82ebc66cf3772f2620641d1e1d271cf7
Aug 09 04:45:17 dist-test-slave-1zqn krb5kdc[11226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:17.742115 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:17.728703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:51207 (local address 127.7.171.62:34801)
0809 04:45:17.728882 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:17.728888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:17.729760 (+ 872us) server_negotiation.cc:407] Connection header received
0809 04:45:17.731164 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:17.731166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:17.731216 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:17.731298 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:17.732565 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.733116 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.734318 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.734539 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.734621 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:17.737556 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:17.737559 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:17.737564 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:17.737601 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:17.739691 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:17.740276 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:17.740282 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:17.740287 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:17.740390 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:17.740757 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:17.740763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:17.740768 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:17.741031 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:17.741144 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:17.741872 (+ 728us) server_negotiation.cc:299] Negotiation successful
0809 04:45:17.741987 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:45:17.743109 11651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:17.729121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34801 (local address 127.7.171.2:51207)
0809 04:45:17.729604 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:17.729648 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:17.730864 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:17.731470 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:17.731476 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:17.731837 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:17.732414 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:17.732427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.733264 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.733267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:17.734200 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:17.734209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.734679 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.734681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:17.734829 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:17.735511 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:17.735528 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:17.737382 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:17.739815 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:17.739825 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:17.739839 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:17.740175 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:17.740512 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:17.740514 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:17.740516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:17.740654 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:17.741163 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:17.741172 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:17.741522 (+ 350us) client_negotiation.cc:769] Sending connection context
0809 04:45:17.741778 (+ 256us) client_negotiation.cc:240] Negotiation successful
0809 04:45:17.742162 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":112,"threads_started":1}
I20260809 04:45:17.744338 11649 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34801
I20260809 04:45:17.744607 11649 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:17.745127 11649 heartbeater.cc:511] Master 127.7.171.62:34801 requested a full tablet report, sending...
I20260809 04:45:17.746284 11280 ts_manager.cc:194] Registered new tserver with Master: b8fcb978fd9e437a911ef25fe21c8a68 (127.7.171.2:39023)
I20260809 04:45:17.746932 11280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:51207
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:45:17.772099 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34801
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:17.890785 11654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:17.891067 11654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:17.891112 11654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:17.895105 11654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:17.895167 11654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:17.895227 11654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:17.900715 11654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:17.901724 11654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:17.902698 11654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:45:17.912739 11654 server_base.cc:1034] running on GCE node
W20260809 04:45:17.912721 11660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:17.912751 11659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:17.912775 11662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:17.913388 11654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:17.914089 11654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:17.915277 11654 hybrid_clock.cc:648] HybridClock initialized: now 1786250717915243 us; error 50 us; skew 500 ppm
Aug 09 04:45:17 dist-test-slave-1zqn krb5kdc[11226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:17.918231 11654 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:45:17.919195 11654 webserver.cc:460] Webserver started at http://127.7.171.3:40221/ using document root <none> and password file <none>
I20260809 04:45:17.919731 11654 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:17.919792 11654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:17.919965 11654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:17.922092 11654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "7ca7d3a258dc4480aa0b4ab7af88e645"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "b80160c5ba52e99a916d67480eb3838a"
server_key_iv: "578bd658a0c98636552200e33789bfc7"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.922719 11654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "7ca7d3a258dc4480aa0b4ab7af88e645"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "b80160c5ba52e99a916d67480eb3838a"
server_key_iv: "578bd658a0c98636552200e33789bfc7"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.926648 11654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 04:45:17.929093 11669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:17.930364 11654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:17.930476 11654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "7ca7d3a258dc4480aa0b4ab7af88e645"
format_stamp: "Formatted at 2026-08-09 04:45:17 on dist-test-slave-1zqn"
server_key: "b80160c5ba52e99a916d67480eb3838a"
server_key_iv: "578bd658a0c98636552200e33789bfc7"
server_key_version: "encryptionkey@0"
I20260809 04:45:17.930563 11654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:17.958195 11654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:17.961750 11654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:17.961912 11654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:17.962481 11654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:17.963286 11654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:17.963335 11654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:17.963377 11654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:17.963393 11654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:17.972488 11654 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:44517
I20260809 04:45:17.972508 11802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:44517 every 8 connection(s)
I20260809 04:45:17.973423 11654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:17.980207 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 11654
I20260809 04:45:17.980350 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:17.980695 7852 external_mini_cluster.cc:1428] Setting key 922b4aef9078c3b0bb474d622499a9a0
Aug 09 04:45:17 dist-test-slave-1zqn krb5kdc[11226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250717, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:17.987564 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:17.975239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:54117 (local address 127.7.171.62:34801)
0809 04:45:17.975380 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:17.975383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:17.976167 (+ 784us) server_negotiation.cc:407] Connection header received
0809 04:45:17.977596 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:17.977598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:17.977649 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:17.977726 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:17.978902 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.979428 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.980733 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.981047 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.981143 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:17.983840 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:17.983843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:17.983845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:17.983873 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:17.985397 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:17.985932 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:17.985936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:17.985937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:17.986034 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:17.986410 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:17.986413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:17.986414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:17.986613 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:17.986704 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:17.987273 (+ 569us) server_negotiation.cc:299] Negotiation successful
0809 04:45:17.987417 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:45:17.988376 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:17.975564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34801 (local address 127.7.171.3:54117)
0809 04:45:17.976013 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:17.976072 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:17.977355 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:17.977893 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:17.977899 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:17.978262 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:17.978766 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:17.978780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.979550 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.979553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:17.980606 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:17.980616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:17.981222 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:17.981228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:17.981417 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:17.982103 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:17.982124 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:17.983658 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:17.985516 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:17.985521 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:17.985533 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:17.985790 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:17.986146 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:17.986149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:17.986150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:17.986286 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:17.986712 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:17.986717 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:17.986958 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 04:45:17.987135 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 04:45:17.987399 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":102,"threads_started":1}
I20260809 04:45:17.989598 11803 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34801
I20260809 04:45:17.989952 11803 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:17.990525 11803 heartbeater.cc:511] Master 127.7.171.62:34801 requested a full tablet report, sending...
I20260809 04:45:17.991641 11280 ts_manager.cc:194] Registered new tserver with Master: 7ca7d3a258dc4480aa0b4ab7af88e645 (127.7.171.3:44517)
I20260809 04:45:17.992321 11280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:54117
I20260809 04:45:17.995437 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:45:18 dist-test-slave-1zqn krb5kdc[11226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250718, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 04:45:18 dist-test-slave-1zqn krb5kdc[11226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250718, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:18.024353 11326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:18.015168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37600 (local address 127.7.171.62:34801)
0809 04:45:18.015341 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:18.015347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:18.015538 (+ 191us) server_negotiation.cc:407] Connection header received
0809 04:45:18.015721 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:18.015725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:18.015781 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:18.015878 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:18.016689 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:18.017221 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:18.018098 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:18.018331 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:18.018444 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:18.021002 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:18.021011 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:18.021014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:18.021041 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:18.022538 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:18.022981 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:18.022985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:18.022987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:18.023073 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:18.023351 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:18.023355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:18.023357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:18.023618 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:18.023764 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:18.024014 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 04:45:18.024182 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 04:45:18.024802 11280 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37600
I20260809 04:45:18.025803 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 11352
I20260809 04:45:18.034698 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 11500
I20260809 04:45:18.048605 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 11654
I20260809 04:45:18.056957 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 11241
2026-08-09T04:45:18Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2356 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:18 dist-test-slave-1zqn krb5kdc[11816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:18 dist-test-slave-1zqn krb5kdc[11816](info): set up 2 sockets
Aug 09 04:45:18 dist-test-slave-1zqn krb5kdc[11816](info): commencing operation
krb5kdc: starting...
W20260809 04:45:19.230887 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:19 dist-test-slave-1zqn krb5kdc[11816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:19.362540 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45949
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:45949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:19.480939 11832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:19.481279 11832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:19.481361 11832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:19.485633 11832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:19.485730 11832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:19.485769 11832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:19.485821 11832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:19.485859 11832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:19.491425 11832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:45949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:19.492683 11832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:19.493786 11832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:19.502785 11838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:19.502800 11837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:19.502800 11840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:19.502875 11832 server_base.cc:1034] running on GCE node
I20260809 04:45:19.503623 11832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:19.504835 11832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:19.506028 11832 hybrid_clock.cc:648] HybridClock initialized: now 1786250719505998 us; error 47 us; skew 500 ppm
Aug 09 04:45:19 dist-test-slave-1zqn krb5kdc[11816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250719, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:19.509032 11832 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:45:19.509904 11832 webserver.cc:460] Webserver started at http://127.7.171.62:33803/ using document root <none> and password file <none>
I20260809 04:45:19.510470 11832 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:19.510562 11832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:19.510823 11832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:19.512981 11832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "fa8490bdbf4941ba96d1f2f27f2d8a54"
format_stamp: "Formatted at 2026-08-09 04:45:19 on dist-test-slave-1zqn"
server_key: "6b574c7bb7c22eb7409fd44550c29122"
server_key_iv: "730bc445b91d9c1aa7739c119fab5f76"
server_key_version: "encryptionkey@0"
I20260809 04:45:19.513566 11832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "fa8490bdbf4941ba96d1f2f27f2d8a54"
format_stamp: "Formatted at 2026-08-09 04:45:19 on dist-test-slave-1zqn"
server_key: "6b574c7bb7c22eb7409fd44550c29122"
server_key_iv: "730bc445b91d9c1aa7739c119fab5f76"
server_key_version: "encryptionkey@0"
I20260809 04:45:19.517459 11832 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:19.519888 11847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:19.521178 11832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:45:19.521337 11832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "fa8490bdbf4941ba96d1f2f27f2d8a54"
format_stamp: "Formatted at 2026-08-09 04:45:19 on dist-test-slave-1zqn"
server_key: "6b574c7bb7c22eb7409fd44550c29122"
server_key_iv: "730bc445b91d9c1aa7739c119fab5f76"
server_key_version: "encryptionkey@0"
I20260809 04:45:19.521453 11832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:19.545609 11832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:19.549278 11832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:19.549502 11832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:19.557551 11832 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:45949
I20260809 04:45:19.557585 11910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:45949 every 8 connection(s)
I20260809 04:45:19.558579 11832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:19.559821 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 11832
I20260809 04:45:19.559955 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:19.560279 7852 external_mini_cluster.cc:1428] Setting key 417d66519de8049d6ab5fe6f7ae8bb08
I20260809 04:45:19.561851 11911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:19.569095 11911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54: Bootstrap starting.
Aug 09 04:45:19 dist-test-slave-1zqn krb5kdc[11816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:19.571971 11911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:19.572805 11911 log.cc:824] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:19.574772 11911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54: No bootstrap required, opened a new log
I20260809 04:45:19.576012 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:19.561833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39732 (local address 127.7.171.62:45949)
0809 04:45:19.562333 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:19.562353 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:19.562398 (+ 45us) server_negotiation.cc:407] Connection header received
0809 04:45:19.563272 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:19.563296 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:19.563727 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:19.564144 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:19.564904 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:19.566463 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:19.567345 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:19.568044 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:19.568266 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:19.571076 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:19.571082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:19.571098 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:19.571141 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:19.573102 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:19.573585 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:19.573592 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:19.573596 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:19.573713 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:19.574007 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:19.574011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:19.574013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:19.574420 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:19.574643 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:19.574930 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 04:45:19.575278 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":176,"threads_started":1}
I20260809 04:45:19.579054 11911 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa8490bdbf4941ba96d1f2f27f2d8a54" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45949 } }
I20260809 04:45:19.579309 11911 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:19.579402 11911 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa8490bdbf4941ba96d1f2f27f2d8a54, State: Initialized, Role: FOLLOWER
I20260809 04:45:19.579880 11911 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa8490bdbf4941ba96d1f2f27f2d8a54" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45949 } }
I20260809 04:45:19.580039 11911 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:19.580164 11911 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:19.580273 11911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:19.581241 11911 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa8490bdbf4941ba96d1f2f27f2d8a54" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45949 } }
I20260809 04:45:19.581632 11911 leader_election.cc:302] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa8490bdbf4941ba96d1f2f27f2d8a54; no voters:
I20260809 04:45:19.581964 11911 leader_election.cc:288] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:19.582119 11916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:19.582434 11916 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [term 1 LEADER]: Becoming Leader. State: Replica: fa8490bdbf4941ba96d1f2f27f2d8a54, State: Running, Role: LEADER
I20260809 04:45:19.582772 11916 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa8490bdbf4941ba96d1f2f27f2d8a54" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45949 } }
I20260809 04:45:19.583434 11911 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:19.584755 11918 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa8490bdbf4941ba96d1f2f27f2d8a54. Latest consensus state: current_term: 1 leader_uuid: "fa8490bdbf4941ba96d1f2f27f2d8a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa8490bdbf4941ba96d1f2f27f2d8a54" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45949 } } }
I20260809 04:45:19.584910 11918 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:19.585347 11925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:19.585321 11917 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa8490bdbf4941ba96d1f2f27f2d8a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa8490bdbf4941ba96d1f2f27f2d8a54" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45949 } } }
I20260809 04:45:19.585999 11917 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:19.588569 11925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:19.594264 11925 catalog_manager.cc:1347] Generated new cluster ID: bf40e49b9fbc4ee1ab17f78a8b1b2e14
I20260809 04:45:19.594343 11925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:19.606868 11925 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:19.608227 11925 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:19.617797 11925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa8490bdbf4941ba96d1f2f27f2d8a54: Generated new TSK 0
I20260809 04:45:19.618690 11925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:45:19.655505 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45949
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:19.773103 11937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:19.773379 11937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:19.773458 11937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:19.777791 11937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:19.777872 11937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:19.777976 11937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:19.783474 11937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:19.784699 11937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:19.785810 11937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:19.795871 11944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:19.795871 11943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:19.795876 11947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:19.796303 11937 server_base.cc:1034] running on GCE node
I20260809 04:45:19.796713 11937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:19.797351 11937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:19.798552 11937 hybrid_clock.cc:648] HybridClock initialized: now 1786250719798522 us; error 42 us; skew 500 ppm
Aug 09 04:45:19 dist-test-slave-1zqn krb5kdc[11816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250719, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:19.801678 11937 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:45:19.802590 11937 webserver.cc:460] Webserver started at http://127.7.171.1:46061/ using document root <none> and password file <none>
I20260809 04:45:19.803140 11937 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:19.803201 11937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:19.803368 11937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:19.805461 11937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "190ca18f33624d7481a40c55a6cd34fa"
format_stamp: "Formatted at 2026-08-09 04:45:19 on dist-test-slave-1zqn"
server_key: "a08249b271ce9e23b585f3f01d9b5af3"
server_key_iv: "39d4ea3d6f301cfcaa01f87c1ff7eaa3"
server_key_version: "encryptionkey@0"
I20260809 04:45:19.805984 11937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "190ca18f33624d7481a40c55a6cd34fa"
format_stamp: "Formatted at 2026-08-09 04:45:19 on dist-test-slave-1zqn"
server_key: "a08249b271ce9e23b585f3f01d9b5af3"
server_key_iv: "39d4ea3d6f301cfcaa01f87c1ff7eaa3"
server_key_version: "encryptionkey@0"
I20260809 04:45:19.809756 11937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:45:19.812211 11955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:19.813493 11937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:19.813653 11937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "190ca18f33624d7481a40c55a6cd34fa"
format_stamp: "Formatted at 2026-08-09 04:45:19 on dist-test-slave-1zqn"
server_key: "a08249b271ce9e23b585f3f01d9b5af3"
server_key_iv: "39d4ea3d6f301cfcaa01f87c1ff7eaa3"
server_key_version: "encryptionkey@0"
I20260809 04:45:19.813783 11937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:19.826097 11937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:19.829157 11937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:19.829375 11937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:19.830000 11937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:19.830871 11937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:19.830945 11937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:19.831018 11937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:19.831073 11937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:19.841328 11937 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:42573
I20260809 04:45:19.841415 12088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:42573 every 8 connection(s)
I20260809 04:45:19.842344 11937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:19.843155 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 11937
I20260809 04:45:19.843312 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:19.843618 7852 external_mini_cluster.cc:1428] Setting key 8aa863985be4b4099fafd9da37b170d9
Aug 09 04:45:19 dist-test-slave-1zqn krb5kdc[11816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250719, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
I20260809 04:45:19.857692 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:19.844468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:34761 (local address 127.7.171.62:45949)
0809 04:45:19.844616 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:19.844621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:19.845709 (+ 1088us) server_negotiation.cc:407] Connection header received
0809 04:45:19.847156 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:19.847159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:19.847218 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:19.847331 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:19.848652 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:19.849224 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:19.850401 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:19.850652 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:19.850767 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:19.853761 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:19.853764 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:19.853766 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:19.853794 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:19.855386 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:19.855907 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:19.855910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:19.855911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:19.855990 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:19.856440 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:19.856443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:19.856444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:19.856649 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:19.856731 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:19.857372 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 04:45:19.857519 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:45:19.858596 12091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:19.844950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45949 (local address 127.7.171.1:34761)
0809 04:45:19.845535 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:19.845598 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:19.846931 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:19.847528 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:19.847536 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:19.847923 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:19.848494 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:19.848509 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:19.849362 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:19.849367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:19.850268 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:19.850278 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:19.850816 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:19.850820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:19.851000 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:19.851662 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:19.851680 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:19.853540 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:19.855505 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:19.855511 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:19.855524 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:19.855813 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:19.856124 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:19.856128 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:19.856130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:19.856316 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:19.856749 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:19.856754 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:19.857084 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 04:45:19.857315 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 04:45:19.857688 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":160,"threads_started":1}
I20260809 04:45:19.859895 12089 heartbeater.cc:347] Connected to a master server at 127.7.171.62:45949
I20260809 04:45:19.860212 12089 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:19.860736 12089 heartbeater.cc:511] Master 127.7.171.62:45949 requested a full tablet report, sending...
I20260809 04:45:19.862332 11867 ts_manager.cc:194] Registered new tserver with Master: 190ca18f33624d7481a40c55a6cd34fa (127.7.171.1:42573)
I20260809 04:45:19.863476 11867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:34761
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:45:19.873518 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45949
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:19.990736 12094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:19.991029 12094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:19.991075 12094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:19.995256 12094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:19.995327 12094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:19.995390 12094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:20.000917 12094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:20.001974 12094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:20.002983 12094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:20.013365 12100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:20.013434 12104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:20.013402 12101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:20.013479 12094 server_base.cc:1034] running on GCE node
I20260809 04:45:20.014070 12094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:20.014684 12094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:20.015846 12094 hybrid_clock.cc:648] HybridClock initialized: now 1786250720015810 us; error 49 us; skew 500 ppm
Aug 09 04:45:20 dist-test-slave-1zqn krb5kdc[11816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250720, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:20.018832 12094 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:45:20.019831 12094 webserver.cc:460] Webserver started at http://127.7.171.2:36179/ using document root <none> and password file <none>
I20260809 04:45:20.020495 12094 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:20.020588 12094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:20.020831 12094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:20.022956 12094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "2db9be61bd2246e8a3ae7aa03cadf4c4"
format_stamp: "Formatted at 2026-08-09 04:45:20 on dist-test-slave-1zqn"
server_key: "09ffd242425e72f083a8de689e8321f8"
server_key_iv: "b8091caea6915f99c5691dcabcfa8273"
server_key_version: "encryptionkey@0"
I20260809 04:45:20.023535 12094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "2db9be61bd2246e8a3ae7aa03cadf4c4"
format_stamp: "Formatted at 2026-08-09 04:45:20 on dist-test-slave-1zqn"
server_key: "09ffd242425e72f083a8de689e8321f8"
server_key_iv: "b8091caea6915f99c5691dcabcfa8273"
server_key_version: "encryptionkey@0"
I20260809 04:45:20.027586 12094 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 04:45:20.030123 12112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:20.031430 12094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:20.031563 12094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "2db9be61bd2246e8a3ae7aa03cadf4c4"
format_stamp: "Formatted at 2026-08-09 04:45:20 on dist-test-slave-1zqn"
server_key: "09ffd242425e72f083a8de689e8321f8"
server_key_iv: "b8091caea6915f99c5691dcabcfa8273"
server_key_version: "encryptionkey@0"
I20260809 04:45:20.031672 12094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:20.048511 12094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:20.051602 12094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:20.051828 12094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:20.052542 12094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:20.053463 12094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:20.053514 12094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:20.053584 12094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:20.053625 12094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:20.062969 12094 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:41167
I20260809 04:45:20.063000 12246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:41167 every 8 connection(s)
I20260809 04:45:20.063942 12094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:20.071555 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 12094
I20260809 04:45:20.071681 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:20.072022 7852 external_mini_cluster.cc:1428] Setting key 23d5f868687458daa982f442b4a90bd2
Aug 09 04:45:20 dist-test-slave-1zqn krb5kdc[11816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250720, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:20.079218 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:20.066029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:48517 (local address 127.7.171.62:45949)
0809 04:45:20.066199 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:20.066202 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:20.067234 (+ 1032us) server_negotiation.cc:407] Connection header received
0809 04:45:20.068688 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:20.068692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:20.068754 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:20.068860 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:20.070058 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.070601 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.071855 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.072249 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.072377 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:20.075086 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:20.075093 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:20.075099 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:20.075141 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:20.076800 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:20.077571 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:20.077575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:20.077577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:20.077678 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:20.078059 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:20.078066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:20.078068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:20.078265 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:20.078379 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:20.078917 (+ 538us) server_negotiation.cc:299] Negotiation successful
0809 04:45:20.079051 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:45:20.080241 12249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:20.066585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45949 (local address 127.7.171.2:48517)
0809 04:45:20.067060 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:20.067114 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:20.068461 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:20.069041 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:20.069049 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:20.069427 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:20.069901 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:20.069914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.070732 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.070737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:20.071694 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:20.071708 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.072413 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.072416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:20.072639 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:20.073288 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:20.073308 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:20.074913 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:20.077058 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:20.077065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:20.077082 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:20.077423 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:20.077783 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:20.077787 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:20.077789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:20.077954 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:20.078379 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:20.078387 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:20.078667 (+ 280us) client_negotiation.cc:769] Sending connection context
0809 04:45:20.078871 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 04:45:20.079234 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":128,"threads_started":1}
I20260809 04:45:20.081701 12247 heartbeater.cc:347] Connected to a master server at 127.7.171.62:45949
I20260809 04:45:20.081977 12247 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:20.082515 12247 heartbeater.cc:511] Master 127.7.171.62:45949 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:45:20.083734 11867 ts_manager.cc:194] Registered new tserver with Master: 2db9be61bd2246e8a3ae7aa03cadf4c4 (127.7.171.2:41167)
I20260809 04:45:20.084571 11867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:48517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:45:20.103842 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45949
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:20.221433 12252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:20.221714 12252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:20.221760 12252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:20.225963 12252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:20.226034 12252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:20.226095 12252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:20.231618 12252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:20.232806 12252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:20.233812 12252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:20.244344 12259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:20.244449 12261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:20.244362 12258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:20.244891 12252 server_base.cc:1034] running on GCE node
I20260809 04:45:20.245428 12252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:20.246217 12252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:20.247432 12252 hybrid_clock.cc:648] HybridClock initialized: now 1786250720247387 us; error 41 us; skew 500 ppm
Aug 09 04:45:20 dist-test-slave-1zqn krb5kdc[11816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250720, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:20.250661 12252 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:45:20.251547 12252 webserver.cc:460] Webserver started at http://127.7.171.3:38475/ using document root <none> and password file <none>
I20260809 04:45:20.252214 12252 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:20.252315 12252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:20.252564 12252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:20.254778 12252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "b98cdec011794a348b80f76bdfded676"
format_stamp: "Formatted at 2026-08-09 04:45:20 on dist-test-slave-1zqn"
server_key: "2f85e9a063541c6bc0723561455fbb16"
server_key_iv: "de4f7e6a9f5f3b6c03d22e2ad2062f2d"
server_key_version: "encryptionkey@0"
I20260809 04:45:20.255394 12252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "b98cdec011794a348b80f76bdfded676"
format_stamp: "Formatted at 2026-08-09 04:45:20 on dist-test-slave-1zqn"
server_key: "2f85e9a063541c6bc0723561455fbb16"
server_key_iv: "de4f7e6a9f5f3b6c03d22e2ad2062f2d"
server_key_version: "encryptionkey@0"
I20260809 04:45:20.259411 12252 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:45:20.262008 12268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:20.263274 12252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:45:20.263414 12252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "b98cdec011794a348b80f76bdfded676"
format_stamp: "Formatted at 2026-08-09 04:45:20 on dist-test-slave-1zqn"
server_key: "2f85e9a063541c6bc0723561455fbb16"
server_key_iv: "de4f7e6a9f5f3b6c03d22e2ad2062f2d"
server_key_version: "encryptionkey@0"
I20260809 04:45:20.263528 12252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:20.281862 12252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:20.285166 12252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:20.285415 12252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:20.286053 12252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:20.286906 12252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:20.286979 12252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:20.287056 12252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:20.287098 12252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:20.296016 12252 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:33547
I20260809 04:45:20.296136 12396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:33547 every 8 connection(s)
I20260809 04:45:20.296999 12252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:20.301388 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 12252
I20260809 04:45:20.301532 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:20.301851 7852 external_mini_cluster.cc:1428] Setting key 05afc38a497e3641ea581f4b6f75913c
Aug 09 04:45:20 dist-test-slave-1zqn krb5kdc[11816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250720, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:20.311405 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:20.298902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:32815 (local address 127.7.171.62:45949)
0809 04:45:20.299051 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:20.299054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:20.300073 (+ 1019us) server_negotiation.cc:407] Connection header received
0809 04:45:20.301835 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:20.301838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:20.301903 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:20.302000 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:20.303476 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.304007 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.305202 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.305427 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.305540 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:20.307981 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:20.307990 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:20.307993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:20.308017 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:20.309324 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:20.309842 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:20.309846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:20.309848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:20.309933 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:20.310271 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:20.310276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:20.310277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:20.310461 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:20.310570 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:20.311105 (+ 535us) server_negotiation.cc:299] Negotiation successful
0809 04:45:20.311231 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:45:20.312412 12399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:20.299342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45949 (local address 127.7.171.3:32815)
0809 04:45:20.299855 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:20.299916 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:20.301613 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:20.302207 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:20.302214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:20.302738 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:20.303338 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:20.303354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.304195 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.304198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:20.305074 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:20.305084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.305592 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.305597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:20.305761 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:20.306311 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:20.306327 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:20.307815 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:20.309446 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:20.309452 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:20.309467 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:20.309735 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:20.310028 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:20.310031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:20.310033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:20.310174 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:20.310581 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:20.310590 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:20.310834 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 04:45:20.311037 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 04:45:20.311363 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":120,"threads_started":1}
I20260809 04:45:20.313515 12397 heartbeater.cc:347] Connected to a master server at 127.7.171.62:45949
I20260809 04:45:20.313748 12397 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:20.314209 12397 heartbeater.cc:511] Master 127.7.171.62:45949 requested a full tablet report, sending...
I20260809 04:45:20.315438 11867 ts_manager.cc:194] Registered new tserver with Master: b98cdec011794a348b80f76bdfded676 (127.7.171.3:33547)
I20260809 04:45:20.315893 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:45:20.316179 11867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:32815
Aug 09 04:45:20 dist-test-slave-1zqn krb5kdc[11816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:45:20 dist-test-slave-1zqn krb5kdc[11816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:45:20.344480 12404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:20.334620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37274 (local address 127.7.171.1:42573)
0809 04:45:20.335033 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:20.335054 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:20.335108 (+ 54us) server_negotiation.cc:407] Connection header received
0809 04:45:20.335183 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:20.335189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:20.335401 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:20.335529 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:20.336286 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.337063 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.337938 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.338325 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.338407 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:20.340919 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:20.340929 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:20.340942 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:20.340967 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:20.342698 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:20.343091 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:20.343096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:20.343097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:20.343179 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:20.343425 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:20.343427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:20.343428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:20.343672 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:20.343844 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:20.344003 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 04:45:20.344246 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":110,"threads_started":1}
Aug 09 04:45:20 dist-test-slave-1zqn krb5kdc[11816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:20.355430 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:20.347465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39746 (local address 127.7.171.62:45949)
0809 04:45:20.347615 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:20.347619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:20.347641 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:45:20.347707 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:20.347709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:20.347749 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:20.347815 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:20.348478 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.349019 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.349767 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:20.349977 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:20.350050 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:20.352566 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:20.352569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:20.352572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:20.352598 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:20.353857 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:20.354252 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:20.354255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:20.354256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:20.354328 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:20.354553 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:20.354555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:20.354556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:20.354792 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:20.354901 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:20.355119 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 04:45:20.355220 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 04:45:20.355913 11867 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39746
I20260809 04:45:20.356637 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 11937
I20260809 04:45:20.365108 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 12094
I20260809 04:45:20.374302 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 12252
I20260809 04:45:20.382473 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 11832
2026-08-09T04:45:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2325 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:20 dist-test-slave-1zqn krb5kdc[12409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:20 dist-test-slave-1zqn krb5kdc[12409](info): set up 2 sockets
Aug 09 04:45:20 dist-test-slave-1zqn krb5kdc[12409](info): commencing operation
krb5kdc: starting...
W20260809 04:45:21.502027 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:21 dist-test-slave-1zqn krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:21.641337 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43271
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:35487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:43271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:21.756181 12426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:21.756498 12426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:21.756544 12426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:21.760833 12426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:21.760912 12426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:21.760941 12426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:21.760987 12426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:21.761029 12426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:21.766583 12426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:43271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:21.767812 12426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:21.768968 12426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:21.777863 12433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:21.777877 12432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:21.778062 12426 server_base.cc:1034] running on GCE node
W20260809 04:45:21.777877 12435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:21.778776 12426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:21.779964 12426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:21.781179 12426 hybrid_clock.cc:648] HybridClock initialized: now 1786250721781163 us; error 28 us; skew 500 ppm
Aug 09 04:45:21 dist-test-slave-1zqn krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250721, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:21.787664 12426 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:45:21.788543 12426 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:21.788617 12426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:21.788795 12426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:21.790957 12426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "de939b011fc8412d8cbb4438358c7a37"
format_stamp: "Formatted at 2026-08-09 04:45:21 on dist-test-slave-1zqn"
server_key: "50ed633da516ae00a2e36dd48c8b7e6a"
server_key_iv: "ebd7e070b0c455fc95dfb9eea4e97a3c"
server_key_version: "encryptionkey@0"
I20260809 04:45:21.791513 12426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "de939b011fc8412d8cbb4438358c7a37"
format_stamp: "Formatted at 2026-08-09 04:45:21 on dist-test-slave-1zqn"
server_key: "50ed633da516ae00a2e36dd48c8b7e6a"
server_key_iv: "ebd7e070b0c455fc95dfb9eea4e97a3c"
server_key_version: "encryptionkey@0"
I20260809 04:45:21.795506 12426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:45:21.798183 12441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:21.799604 12426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:21.799724 12426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "de939b011fc8412d8cbb4438358c7a37"
format_stamp: "Formatted at 2026-08-09 04:45:21 on dist-test-slave-1zqn"
server_key: "50ed633da516ae00a2e36dd48c8b7e6a"
server_key_iv: "ebd7e070b0c455fc95dfb9eea4e97a3c"
server_key_version: "encryptionkey@0"
I20260809 04:45:21.799875 12426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:21.813714 12426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:21.817991 12426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:21.818229 12426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:21.826215 12426 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:43271
I20260809 04:45:21.826517 12500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:43271 every 8 connection(s)
I20260809 04:45:21.827010 12426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:21.828802 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 12426
I20260809 04:45:21.828959 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:21.829268 7852 external_mini_cluster.cc:1428] Setting key 7ac749178f3c842a88c947fea6a15440
I20260809 04:45:21.830295 12501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:21.836388 12501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37: Bootstrap starting.
Aug 09 04:45:21 dist-test-slave-1zqn krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:21.839018 12501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:21.840008 12501 log.cc:824] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:21.842648 12501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37: No bootstrap required, opened a new log
I20260809 04:45:21.844390 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:21.830705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.7.171.62:43271)
0809 04:45:21.831190 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:21.831207 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:21.831245 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:45:21.831944 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:21.831968 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:21.832402 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:21.832761 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:21.833620 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:21.834622 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:21.835507 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:21.836005 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:21.836228 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:21.838924 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:21.838930 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:21.838946 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:21.838978 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:21.841157 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:21.841694 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:21.841699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:21.841705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:21.841841 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:21.842152 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:21.842155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:21.842157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:21.842640 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:21.842857 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:21.843226 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 04:45:21.843547 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":160,"threads_started":1}
I20260809 04:45:21.846832 12501 raft_consensus.cc:348] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de939b011fc8412d8cbb4438358c7a37" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43271 } }
I20260809 04:45:21.847106 12501 raft_consensus.cc:374] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:21.847182 12501 raft_consensus.cc:729] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de939b011fc8412d8cbb4438358c7a37, State: Initialized, Role: FOLLOWER
I20260809 04:45:21.847613 12501 consensus_queue.cc:260] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de939b011fc8412d8cbb4438358c7a37" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43271 } }
I20260809 04:45:21.847730 12501 raft_consensus.cc:388] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:21.847818 12501 raft_consensus.cc:482] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:21.847915 12501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:21.848975 12501 raft_consensus.cc:504] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de939b011fc8412d8cbb4438358c7a37" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43271 } }
I20260809 04:45:21.849341 12501 leader_election.cc:302] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de939b011fc8412d8cbb4438358c7a37; no voters:
I20260809 04:45:21.849710 12501 leader_election.cc:288] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:21.849879 12507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:21.850140 12507 raft_consensus.cc:686] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [term 1 LEADER]: Becoming Leader. State: Replica: de939b011fc8412d8cbb4438358c7a37, State: Running, Role: LEADER
I20260809 04:45:21.850454 12507 consensus_queue.cc:237] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de939b011fc8412d8cbb4438358c7a37" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43271 } }
I20260809 04:45:21.851038 12501 sys_catalog.cc:564] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:21.852680 12508 sys_catalog.cc:455] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de939b011fc8412d8cbb4438358c7a37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de939b011fc8412d8cbb4438358c7a37" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43271 } } }
I20260809 04:45:21.852865 12509 sys_catalog.cc:455] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de939b011fc8412d8cbb4438358c7a37. Latest consensus state: current_term: 1 leader_uuid: "de939b011fc8412d8cbb4438358c7a37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de939b011fc8412d8cbb4438358c7a37" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43271 } } }
I20260809 04:45:21.852959 12509 sys_catalog.cc:458] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:21.852902 12508 sys_catalog.cc:458] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37 [sys.catalog]: This master's current role is: LEADER
W20260809 04:45:21.856230 12525 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:45:21.856323 12525 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:45:21.856408 12522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:21.857218 12522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:21.863202 12522 catalog_manager.cc:1347] Generated new cluster ID: ed0f5af084a248fa9ff34c3c416e0098
I20260809 04:45:21.863278 12522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:21.897619 12522 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:21.898692 12522 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:21.906705 12522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37: Generated new TSK 0
I20260809 04:45:21.907490 12522 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:45:21.947007 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43271
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:35487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:22.088572 12531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:22.088897 12531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:22.088959 12531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:22.093477 12531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:22.093551 12531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:22.093616 12531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:22.099097 12531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:22.100207 12531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:22.101382 12531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:45:22.111935 12531 server_base.cc:1034] running on GCE node
W20260809 04:45:22.111824 12541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:22.111828 12537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:22.111824 12538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:22.112571 12531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:22.113299 12531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:22.114495 12531 hybrid_clock.cc:648] HybridClock initialized: now 1786250722114471 us; error 37 us; skew 500 ppm
Aug 09 04:45:22 dist-test-slave-1zqn krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250722, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:22.117514 12531 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:45:22.118260 12531 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:22.118324 12531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:22.118495 12531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:22.120797 12531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "67588ac5c0454bf8b16676e0e4d45851"
format_stamp: "Formatted at 2026-08-09 04:45:22 on dist-test-slave-1zqn"
server_key: "7f1afc3c4be7ef567fdb164b6c67867d"
server_key_iv: "ed0888cfefbc12ab9374b2563b28330f"
server_key_version: "encryptionkey@0"
I20260809 04:45:22.121341 12531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "67588ac5c0454bf8b16676e0e4d45851"
format_stamp: "Formatted at 2026-08-09 04:45:22 on dist-test-slave-1zqn"
server_key: "7f1afc3c4be7ef567fdb164b6c67867d"
server_key_iv: "ed0888cfefbc12ab9374b2563b28330f"
server_key_version: "encryptionkey@0"
I20260809 04:45:22.125159 12531 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:45:22.127611 12547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:22.128937 12531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 04:45:22.129042 12531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "67588ac5c0454bf8b16676e0e4d45851"
format_stamp: "Formatted at 2026-08-09 04:45:22 on dist-test-slave-1zqn"
server_key: "7f1afc3c4be7ef567fdb164b6c67867d"
server_key_iv: "ed0888cfefbc12ab9374b2563b28330f"
server_key_version: "encryptionkey@0"
I20260809 04:45:22.129132 12531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:22.164448 12531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:22.167798 12531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:22.167966 12531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:22.168643 12531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:22.169469 12531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:22.169525 12531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:22.169567 12531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:22.169588 12531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:22.178645 12531 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:42933
I20260809 04:45:22.178663 12672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:42933 every 8 connection(s)
I20260809 04:45:22.179267 12531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:22.186347 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 12531
I20260809 04:45:22.186496 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:22.186827 7852 external_mini_cluster.cc:1428] Setting key 5530d61661cdc57c55f13c61464dac57
Aug 09 04:45:22 dist-test-slave-1zqn krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250722, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:22.194173 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.181110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:52231 (local address 127.7.171.62:43271)
0809 04:45:22.181262 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:22.181266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:22.182601 (+ 1335us) server_negotiation.cc:407] Connection header received
0809 04:45:22.183497 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:22.183500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:22.183557 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:22.183642 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:22.184971 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.185540 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.187101 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.187380 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.187564 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.190383 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:22.190387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:22.190389 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:22.190412 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:22.191947 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:22.192513 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:22.192517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:22.192518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:22.192613 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:22.192985 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:22.192988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:22.192990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:22.193186 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:22.193296 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:22.193845 (+ 549us) server_negotiation.cc:299] Negotiation successful
0809 04:45:22.193989 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:45:22.195125 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.181498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43271 (local address 127.7.171.1:52231)
0809 04:45:22.182443 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:22.182487 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:22.183274 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:22.183816 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:22.183823 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:22.184312 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:22.184816 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.184830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.185667 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.185669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.186936 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.186949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.187596 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.187599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.187822 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.188604 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:22.188621 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:22.190198 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:22.192096 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:22.192103 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:22.192120 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:22.192401 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:22.192712 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:22.192715 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:22.192718 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:22.192869 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:22.193312 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:22.193320 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:22.193589 (+ 269us) client_negotiation.cc:769] Sending connection context
0809 04:45:22.193783 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 04:45:22.194143 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":131,"threads_started":1}
I20260809 04:45:22.196535 12673 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43271
I20260809 04:45:22.196810 12673 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:45:22.197337 12673 heartbeater.cc:511] Master 127.7.171.62:43271 requested a full tablet report, sending...
I20260809 04:45:22.198980 12462 ts_manager.cc:194] Registered new tserver with Master: 67588ac5c0454bf8b16676e0e4d45851 (127.7.171.1:42933)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
I20260809 04:45:22.200315 12462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:52231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:45:22.219797 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43271
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:35487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:22.339665 12679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:22.339933 12679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:22.339978 12679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:22.343945 12679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:22.344008 12679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:22.344094 12679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:22.349352 12679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:22.350374 12679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:22.351428 12679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:22.361725 12685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:22.361737 12686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:22.361855 12679 server_base.cc:1034] running on GCE node
W20260809 04:45:22.361737 12688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:22.362630 12679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:22.363376 12679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:22.364606 12679 hybrid_clock.cc:648] HybridClock initialized: now 1786250722364582 us; error 38 us; skew 500 ppm
Aug 09 04:45:22 dist-test-slave-1zqn krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250722, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:22.367931 12679 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:45:22.368765 12679 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:22.368835 12679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:22.369025 12679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:22.371359 12679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "0c82afa644524890adfac534ea0448e5"
format_stamp: "Formatted at 2026-08-09 04:45:22 on dist-test-slave-1zqn"
server_key: "c8f4d97fb299d4a8615316be62a75964"
server_key_iv: "e4ee686aa615c98ea3d38aec564b46e7"
server_key_version: "encryptionkey@0"
I20260809 04:45:22.371928 12679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "0c82afa644524890adfac534ea0448e5"
format_stamp: "Formatted at 2026-08-09 04:45:22 on dist-test-slave-1zqn"
server_key: "c8f4d97fb299d4a8615316be62a75964"
server_key_iv: "e4ee686aa615c98ea3d38aec564b46e7"
server_key_version: "encryptionkey@0"
I20260809 04:45:22.376179 12679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:45:22.379040 12696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:22.380534 12679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:22.380671 12679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "0c82afa644524890adfac534ea0448e5"
format_stamp: "Formatted at 2026-08-09 04:45:22 on dist-test-slave-1zqn"
server_key: "c8f4d97fb299d4a8615316be62a75964"
server_key_iv: "e4ee686aa615c98ea3d38aec564b46e7"
server_key_version: "encryptionkey@0"
I20260809 04:45:22.380787 12679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:22.406227 12679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:22.409891 12679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:22.410127 12679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:22.410772 12679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:22.411679 12679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:22.411731 12679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:22.411804 12679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:22.411845 12679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:22.421563 12679 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:42393
I20260809 04:45:22.421586 12822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:42393 every 8 connection(s)
I20260809 04:45:22.422307 12679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:22.428267 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 12679
I20260809 04:45:22.428400 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:22.428689 7852 external_mini_cluster.cc:1428] Setting key e2def35598b3fe824b793c94488d734e
Aug 09 04:45:22 dist-test-slave-1zqn krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250722, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:22.438611 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.424420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:48991 (local address 127.7.171.62:43271)
0809 04:45:22.424587 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:22.424592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:22.425875 (+ 1283us) server_negotiation.cc:407] Connection header received
0809 04:45:22.426715 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:22.426718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:22.426775 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:22.426863 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:22.428184 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.428875 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.430321 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.430557 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.430644 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.434129 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:22.434135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:22.434138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:22.434173 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:22.436128 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:22.436740 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:22.436745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:22.436747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:22.436851 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:22.437209 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:22.437217 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:22.437220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:22.437537 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:22.437667 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:22.438168 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 04:45:22.438344 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 04:45:22.439287 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.424854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43271 (local address 127.7.171.2:48991)
0809 04:45:22.425694 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:22.425738 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:22.426530 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:22.427030 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:22.427037 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:22.427425 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:22.427904 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.427918 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.429016 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.429019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.430197 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.430208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.430725 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.430731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.430960 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.431678 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:22.431704 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:22.433930 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:22.436254 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:22.436259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:22.436272 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:22.436624 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:22.436957 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:22.436959 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:22.436960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:22.437104 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:22.437651 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:22.437656 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:22.437906 (+ 250us) client_negotiation.cc:769] Sending connection context
0809 04:45:22.438073 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 04:45:22.438346 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":135,"threads_started":1}
I20260809 04:45:22.440714 12824 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43271
I20260809 04:45:22.441043 12824 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:45:22.441670 12824 heartbeater.cc:511] Master 127.7.171.62:43271 requested a full tablet report, sending...
I20260809 04:45:22.442912 12462 ts_manager.cc:194] Registered new tserver with Master: 0c82afa644524890adfac534ea0448e5 (127.7.171.2:42393)
I20260809 04:45:22.443519 12462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:48991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:45:22.463244 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43271
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:35487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:22.582438 12829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:22.582868 12829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:22.582948 12829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:22.587805 12829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:22.587914 12829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:22.588014 12829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:22.593876 12829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:22.595131 12829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:22.596289 12829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:22.606419 12839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:22.606498 12836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:22.606546 12835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:22.606827 12829 server_base.cc:1034] running on GCE node
I20260809 04:45:22.607230 12829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:22.607882 12829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:22.609076 12829 hybrid_clock.cc:648] HybridClock initialized: now 1786250722609060 us; error 63 us; skew 500 ppm
Aug 09 04:45:22 dist-test-slave-1zqn krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250722, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:22.611945 12829 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:45:22.612706 12829 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:22.612772 12829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:22.612937 12829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:22.614972 12829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "2a11992260104fcda9368d5833bd1d81"
format_stamp: "Formatted at 2026-08-09 04:45:22 on dist-test-slave-1zqn"
server_key: "eedb8313df97488ff0a136e37377110a"
server_key_iv: "0b23674cc8443f89e03226d218747346"
server_key_version: "encryptionkey@0"
I20260809 04:45:22.615499 12829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "2a11992260104fcda9368d5833bd1d81"
format_stamp: "Formatted at 2026-08-09 04:45:22 on dist-test-slave-1zqn"
server_key: "eedb8313df97488ff0a136e37377110a"
server_key_iv: "0b23674cc8443f89e03226d218747346"
server_key_version: "encryptionkey@0"
I20260809 04:45:22.619267 12829 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:45:22.621824 12846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:22.623136 12829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:22.623279 12829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "2a11992260104fcda9368d5833bd1d81"
format_stamp: "Formatted at 2026-08-09 04:45:22 on dist-test-slave-1zqn"
server_key: "eedb8313df97488ff0a136e37377110a"
server_key_iv: "0b23674cc8443f89e03226d218747346"
server_key_version: "encryptionkey@0"
I20260809 04:45:22.623404 12829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:22.639787 12829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:22.642822 12829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:22.643035 12829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:22.643644 12829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:22.644573 12829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:22.644622 12829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:22.644692 12829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:22.644733 12829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:22.653789 12829 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:38503
I20260809 04:45:22.653947 12971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:38503 every 8 connection(s)
I20260809 04:45:22.654564 12829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:22.660748 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 12829
I20260809 04:45:22.660885 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:22.661193 7852 external_mini_cluster.cc:1428] Setting key c4f1a939f5bd62a5da8b1cc9595d3b20
Aug 09 04:45:22 dist-test-slave-1zqn krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250722, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:22.668900 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.656448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:40513 (local address 127.7.171.62:43271)
0809 04:45:22.656620 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:22.656624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:22.657855 (+ 1231us) server_negotiation.cc:407] Connection header received
0809 04:45:22.658779 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:22.658782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:22.658846 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:22.658927 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:22.660144 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.660833 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.662404 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.662688 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.662781 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.665524 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:22.665527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:22.665529 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:22.665553 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:22.666962 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:22.667461 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:22.667464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:22.667465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:22.667539 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:22.667842 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:22.667844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:22.667845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:22.668015 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:22.668204 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:22.668675 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 04:45:22.668777 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:45:22.669775 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.656866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43271 (local address 127.7.171.3:40513)
0809 04:45:22.657702 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:22.657743 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:22.658545 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:22.659108 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:22.659115 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:22.659479 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:22.659966 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.659979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.660987 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.660993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.662275 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.662285 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.662850 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.662854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.663056 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.663724 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:22.663741 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:22.665371 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:22.667088 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:22.667094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:22.667105 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:22.667367 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:22.667621 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:22.667623 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:22.667625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:22.667760 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:22.668200 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:22.668205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:22.668452 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 04:45:22.668625 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 04:45:22.668893 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":111,"threads_started":1}
I20260809 04:45:22.670992 12972 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43271
I20260809 04:45:22.671248 12972 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:22.671707 12972 heartbeater.cc:511] Master 127.7.171.62:43271 requested a full tablet report, sending...
I20260809 04:45:22.672798 12462 ts_manager.cc:194] Registered new tserver with Master: 2a11992260104fcda9368d5833bd1d81 (127.7.171.3:38503)
I20260809 04:45:22.673488 12462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:40513
I20260809 04:45:22.675311 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:45:22.685295 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.677311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.7.171.62:43271)
0809 04:45:22.677425 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:22.677429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:22.677629 (+ 200us) server_negotiation.cc:407] Connection header received
0809 04:45:22.677737 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:22.677739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:22.677774 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:22.677840 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:22.678626 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.679122 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.680011 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.680301 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.680421 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.681936 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:22.681940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:22.681943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:22.681974 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:22.683481 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:22.683919 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:22.683923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:22.683924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:22.684011 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:22.684351 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:22.684355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:22.684357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:22.684588 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:22.684733 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:22.684971 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 04:45:22.685124 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 04:45:22.688566 12462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:22.691614 12462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:22.707402 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.701201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.7.171.2:42393)
0809 04:45:22.702291 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:22.702302 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:22.702321 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:45:22.702386 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:22.702390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:22.702552 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:22.702701 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:22.703157 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.704528 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.705990 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.706784 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.706978 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.707087 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:22.707131 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 04:45:22.707203 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":994,"thread_start_us":155,"threads_started":1}
I20260809 04:45:22.707424 12987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.700880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:42393 (local address 127.0.0.1:58732)
0809 04:45:22.701642 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:22.701657 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:22.701739 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:22.702682 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:22.702688 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:22.702730 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:22.703008 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.703016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.704708 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.704711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.705832 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.705845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.706919 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.706923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.707025 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.707051 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:45:22.707105 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 04:45:22.707175 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":121,"spinlock_wait_cycles":6656,"thread_start_us":114,"threads_started":1}
I20260809 04:45:22.707764 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.701845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41570 (local address 127.7.171.1:42933)
0809 04:45:22.702307 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:22.702313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:22.702336 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:45:22.702415 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:22.702423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:22.702632 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:22.703165 (+ 533us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:22.703308 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.704528 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.706006 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.706682 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.707267 (+ 585us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.707411 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:22.707477 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 04:45:22.707580 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":123,"threads_started":1}
I20260809 04:45:22.707430 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.701606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:42933 (local address 127.0.0.1:41570)
0809 04:45:22.701918 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:22.701929 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:22.702041 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:22.702780 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:22.702783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:22.702808 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:22.703015 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.703021 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.704653 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.704656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.705883 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.705891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.706829 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.706833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.706913 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.706956 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 04:45:22.707044 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:45:22.707175 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":117,"spinlock_wait_cycles":6016,"thread_start_us":102,"threads_started":1}
I20260809 04:45:22.708457 12986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.700870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:38503 (local address 127.0.0.1:50460)
0809 04:45:22.701242 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:22.701299 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:22.701471 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:22.702678 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:22.702684 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:22.702725 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:22.703014 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.703026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.704207 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.704211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.705623 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.705634 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.706766 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.706771 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:22.706899 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.708155 (+ 1256us) client_negotiation.cc:769] Sending connection context
0809 04:45:22.708249 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 04:45:22.708329 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":116,"spinlock_wait_cycles":14720,"thread_start_us":75,"threads_started":1}
I20260809 04:45:22.709034 12989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.701588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50460 (local address 127.7.171.3:38503)
0809 04:45:22.702150 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:22.702164 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:22.702191 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:45:22.702269 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:22.702274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:22.702492 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:22.702978 (+ 486us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:22.703155 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.703964 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.705754 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.706632 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:22.706755 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:22.706928 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:22.708834 (+ 1906us) server_negotiation.cc:299] Negotiation successful
0809 04:45:22.708920 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":103,"threads_started":1}
I20260809 04:45:22.710470 12748 tablet_service.cc:1467] Processing CreateTablet for tablet 0f966c36dad245038afad5a1b4a3420e (DEFAULT_TABLE table=test-table [id=ea4985b315bf45259bd9495ae2c09af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:22.711472 12748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f966c36dad245038afad5a1b4a3420e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:22.711220 12899 tablet_service.cc:1467] Processing CreateTablet for tablet 0f966c36dad245038afad5a1b4a3420e (DEFAULT_TABLE table=test-table [id=ea4985b315bf45259bd9495ae2c09af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:22.712224 12899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f966c36dad245038afad5a1b4a3420e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:22.717887 12992 tablet_bootstrap.cc:492] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: Bootstrap starting.
I20260809 04:45:22.718947 12597 tablet_service.cc:1467] Processing CreateTablet for tablet 0f966c36dad245038afad5a1b4a3420e (DEFAULT_TABLE table=test-table [id=ea4985b315bf45259bd9495ae2c09af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:22.719820 12597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f966c36dad245038afad5a1b4a3420e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:22.720203 12992 tablet_bootstrap.cc:654] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:22.720620 12993 tablet_bootstrap.cc:492] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: Bootstrap starting.
I20260809 04:45:22.721323 12992 log.cc:824] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:22.722852 12993 tablet_bootstrap.cc:654] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:22.723189 12992 tablet_bootstrap.cc:492] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: No bootstrap required, opened a new log
I20260809 04:45:22.723402 12992 ts_tablet_manager.cc:1397] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 04:45:22.724164 12993 log.cc:824] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:22.726275 12993 tablet_bootstrap.cc:492] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: No bootstrap required, opened a new log
I20260809 04:45:22.726465 12993 ts_tablet_manager.cc:1397] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:45:22.727186 12996 tablet_bootstrap.cc:492] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851: Bootstrap starting.
I20260809 04:45:22.727227 12992 raft_consensus.cc:348] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 } } peers { permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 } } peers { permanent_uuid: "67588ac5c0454bf8b16676e0e4d45851" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42933 } }
I20260809 04:45:22.727514 12992 raft_consensus.cc:374] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:22.727581 12992 raft_consensus.cc:729] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a11992260104fcda9368d5833bd1d81, State: Initialized, Role: FOLLOWER
I20260809 04:45:22.728080 12992 consensus_queue.cc:260] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 } } peers { permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 } } peers { permanent_uuid: "67588ac5c0454bf8b16676e0e4d45851" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42933 } }
I20260809 04:45:22.728834 12972 heartbeater.cc:503] Master 127.7.171.62:43271 was elected leader, sending a full tablet report...
I20260809 04:45:22.729192 12992 ts_tablet_manager.cc:1428] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:45:22.730038 12996 tablet_bootstrap.cc:654] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:22.730494 12993 raft_consensus.cc:348] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 } } peers { permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 } } peers { permanent_uuid: "67588ac5c0454bf8b16676e0e4d45851" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42933 } }
I20260809 04:45:22.730737 12993 raft_consensus.cc:374] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:22.730793 12993 raft_consensus.cc:729] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c82afa644524890adfac534ea0448e5, State: Initialized, Role: FOLLOWER
I20260809 04:45:22.730906 12996 log.cc:824] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:22.731238 12993 consensus_queue.cc:260] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 } } peers { permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 } } peers { permanent_uuid: "67588ac5c0454bf8b16676e0e4d45851" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42933 } }
I20260809 04:45:22.731904 12824 heartbeater.cc:503] Master 127.7.171.62:43271 was elected leader, sending a full tablet report...
I20260809 04:45:22.732311 12993 ts_tablet_manager.cc:1428] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 04:45:22.734604 12996 tablet_bootstrap.cc:492] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851: No bootstrap required, opened a new log
I20260809 04:45:22.734833 12996 ts_tablet_manager.cc:1397] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260809 04:45:22.738490 12996 raft_consensus.cc:348] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 } } peers { permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 } } peers { permanent_uuid: "67588ac5c0454bf8b16676e0e4d45851" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42933 } }
I20260809 04:45:22.738766 12996 raft_consensus.cc:374] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:22.738831 12996 raft_consensus.cc:729] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67588ac5c0454bf8b16676e0e4d45851, State: Initialized, Role: FOLLOWER
I20260809 04:45:22.739260 12996 consensus_queue.cc:260] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 } } peers { permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 } } peers { permanent_uuid: "67588ac5c0454bf8b16676e0e4d45851" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42933 } }
I20260809 04:45:22.740046 12673 heartbeater.cc:503] Master 127.7.171.62:43271 was elected leader, sending a full tablet report...
I20260809 04:45:22.740283 12996 ts_tablet_manager.cc:1428] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
W20260809 04:45:22.905547 12973 tablet.cc:2369] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:45:22.923609 12825 tablet.cc:2369] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:45:22.930634 12674 tablet.cc:2369] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:45:22.996764 13000 raft_consensus.cc:482] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:22.996974 13000 raft_consensus.cc:504] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 } } peers { permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 } } peers { permanent_uuid: "67588ac5c0454bf8b16676e0e4d45851" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42933 } }
I20260809 04:45:22.998142 13000 leader_election.cc:288] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a11992260104fcda9368d5833bd1d81 (127.7.171.3:38503), 0c82afa644524890adfac534ea0448e5 (127.7.171.2:42393)
I20260809 04:45:23.002184 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.998499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:42687 (local address 127.7.171.2:42393)
0809 04:45:22.998660 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:22.998665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:22.998964 (+ 299us) server_negotiation.cc:407] Connection header received
0809 04:45:22.999079 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:22.999081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:22.999155 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:22.999235 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:22.999533 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.000008 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.001199 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.001613 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.001735 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:23.001801 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:23.001981 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 04:45:23.002054 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:45:23.002207 13002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.998391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:42393 (local address 127.7.171.1:42687)
0809 04:45:22.998844 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:22.998860 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:22.998975 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:22.999275 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:22.999277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:22.999291 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:22.999384 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.999388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.000179 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.000182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:23.001073 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:23.001082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.001745 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.001748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:23.001856 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:23.001876 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:45:23.001964 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:45:23.002046 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":38,"thread_start_us":133,"threads_started":1}
I20260809 04:45:23.002249 12989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.998503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:35897 (local address 127.7.171.3:38503)
0809 04:45:22.998657 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:22.998662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:22.998874 (+ 212us) server_negotiation.cc:407] Connection header received
0809 04:45:22.998990 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:22.998995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:22.999060 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:22.999133 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:22.999466 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:22.999987 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.001235 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.001682 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.001799 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:23.001863 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:23.002065 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 04:45:23.002136 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:45:23.002266 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:22.998372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:38503 (local address 127.7.171.1:35897)
0809 04:45:22.998767 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:22.998779 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:22.998876 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:22.999170 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:22.999176 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:22.999198 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:22.999348 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:22.999354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.000151 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.000154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:23.001076 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:23.001085 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.001814 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.001818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:23.001937 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:23.001955 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 04:45:23.002049 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 04:45:23.002119 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":5,"thread_start_us":126,"threads_started":1}
I20260809 04:45:23.002805 12997 raft_consensus.cc:482] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:23.002990 12997 raft_consensus.cc:504] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 } } peers { permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 } } peers { permanent_uuid: "67588ac5c0454bf8b16676e0e4d45851" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42933 } }
I20260809 04:45:23.003039 12771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f966c36dad245038afad5a1b4a3420e" candidate_uuid: "67588ac5c0454bf8b16676e0e4d45851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c82afa644524890adfac534ea0448e5" is_pre_election: true
I20260809 04:45:23.003049 12919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f966c36dad245038afad5a1b4a3420e" candidate_uuid: "67588ac5c0454bf8b16676e0e4d45851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a11992260104fcda9368d5833bd1d81" is_pre_election: true
I20260809 04:45:23.003334 12771 raft_consensus.cc:2454] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67588ac5c0454bf8b16676e0e4d45851 in term 0.
I20260809 04:45:23.003477 12919 raft_consensus.cc:2454] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67588ac5c0454bf8b16676e0e4d45851 in term 0.
I20260809 04:45:23.003845 12549 leader_election.cc:302] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c82afa644524890adfac534ea0448e5, 67588ac5c0454bf8b16676e0e4d45851; no voters:
I20260809 04:45:23.004194 13000 raft_consensus.cc:2781] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:23.004284 13000 raft_consensus.cc:482] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:23.004328 13000 raft_consensus.cc:3037] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:23.004359 12997 leader_election.cc:288] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c82afa644524890adfac534ea0448e5 (127.7.171.2:42393), 67588ac5c0454bf8b16676e0e4d45851 (127.7.171.1:42933)
I20260809 04:45:23.005815 13000 raft_consensus.cc:504] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 } } peers { permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 } } peers { permanent_uuid: "67588ac5c0454bf8b16676e0e4d45851" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42933 } }
I20260809 04:45:23.006196 13000 leader_election.cc:288] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [CANDIDATE]: Term 1 election: Requested vote from peers 2a11992260104fcda9368d5833bd1d81 (127.7.171.3:38503), 0c82afa644524890adfac534ea0448e5 (127.7.171.2:42393)
I20260809 04:45:23.006534 12919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f966c36dad245038afad5a1b4a3420e" candidate_uuid: "67588ac5c0454bf8b16676e0e4d45851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a11992260104fcda9368d5833bd1d81"
I20260809 04:45:23.006664 12919 raft_consensus.cc:3037] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:23.007001 12771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f966c36dad245038afad5a1b4a3420e" candidate_uuid: "67588ac5c0454bf8b16676e0e4d45851" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c82afa644524890adfac534ea0448e5"
I20260809 04:45:23.007133 12771 raft_consensus.cc:3037] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:23.007953 12919 raft_consensus.cc:2454] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67588ac5c0454bf8b16676e0e4d45851 in term 1.
I20260809 04:45:23.008332 12548 leader_election.cc:302] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a11992260104fcda9368d5833bd1d81, 67588ac5c0454bf8b16676e0e4d45851; no voters:
I20260809 04:45:23.008494 13000 raft_consensus.cc:2781] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:23.008498 12771 raft_consensus.cc:2454] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67588ac5c0454bf8b16676e0e4d45851 in term 1.
I20260809 04:45:23.008685 13000 raft_consensus.cc:686] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [term 1 LEADER]: Becoming Leader. State: Replica: 67588ac5c0454bf8b16676e0e4d45851, State: Running, Role: LEADER
I20260809 04:45:23.008852 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.004523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:42393 (local address 127.7.171.3:59325)
0809 04:45:23.004657 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:23.004674 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:23.004755 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:23.005017 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:23.005020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:23.005047 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:23.005170 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:23.005180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.006114 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.006120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:23.007338 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:23.007350 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.008359 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.008366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:23.008498 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:23.008526 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 04:45:23.008619 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 04:45:23.008703 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":28}
I20260809 04:45:23.009068 13000 consensus_queue.cc:237] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 } } peers { permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 } } peers { permanent_uuid: "67588ac5c0454bf8b16676e0e4d45851" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42933 } }
I20260809 04:45:23.009312 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.004679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:59325 (local address 127.7.171.2:42393)
0809 04:45:23.004809 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:23.004813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:23.004825 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:23.004863 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:23.004866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:23.004904 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:23.004984 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:23.005304 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.006010 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.007483 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.008133 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.008234 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:23.008294 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:23.009111 (+ 817us) server_negotiation.cc:299] Negotiation successful
0809 04:45:23.009187 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:45:23.009894 12771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f966c36dad245038afad5a1b4a3420e" candidate_uuid: "2a11992260104fcda9368d5833bd1d81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c82afa644524890adfac534ea0448e5" is_pre_election: true
I20260809 04:45:23.009970 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.005240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:47685 (local address 127.7.171.1:42933)
0809 04:45:23.005364 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:23.005368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:23.005405 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:45:23.005530 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:23.005533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:23.005583 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:23.005659 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:23.005912 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.006599 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.008291 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.008892 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.008990 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:23.009042 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:23.009776 (+ 734us) server_negotiation.cc:299] Negotiation successful
0809 04:45:23.009847 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:45:23.010115 12771 raft_consensus.cc:2379] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a11992260104fcda9368d5833bd1d81 in current term 1: Already voted for candidate 67588ac5c0454bf8b16676e0e4d45851 in this term.
I20260809 04:45:23.010282 13003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.004829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:42933 (local address 127.7.171.3:47685)
0809 04:45:23.005335 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:23.005348 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:23.005437 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:23.005681 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:23.005684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:23.005701 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:23.005802 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:23.005807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.006859 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.006863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:23.008156 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:23.008169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.009258 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.009263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:23.009377 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:23.009655 (+ 278us) client_negotiation.cc:769] Sending connection context
0809 04:45:23.010083 (+ 428us) client_negotiation.cc:240] Negotiation successful
0809 04:45:23.010156 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":28,"thread_start_us":139,"threads_started":1}
I20260809 04:45:23.011301 12620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f966c36dad245038afad5a1b4a3420e" candidate_uuid: "2a11992260104fcda9368d5833bd1d81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67588ac5c0454bf8b16676e0e4d45851" is_pre_election: true
I20260809 04:45:23.011878 12849 leader_election.cc:302] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a11992260104fcda9368d5833bd1d81; no voters: 0c82afa644524890adfac534ea0448e5, 67588ac5c0454bf8b16676e0e4d45851
I20260809 04:45:23.012207 12997 raft_consensus.cc:2726] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 04:45:23.012768 12462 catalog_manager.cc:5495] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 reported cstate change: term changed from 0 to 1, leader changed from <none> to 67588ac5c0454bf8b16676e0e4d45851 (127.7.171.1). New cstate: current_term: 1 leader_uuid: "67588ac5c0454bf8b16676e0e4d45851" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67588ac5c0454bf8b16676e0e4d45851" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 42933 } health_report { overall_health: HEALTHY } } }
I20260809 04:45:23.076013 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.072316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41582 (local address 127.7.171.1:42933)
0809 04:45:23.072484 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:23.072491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:23.072507 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:45:23.072597 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:23.072600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:23.072659 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:23.072778 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:23.073043 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.073551 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.074420 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:23.074647 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:23.074764 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:23.075136 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:23.075612 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:23.075756 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:23.075798 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 04:45:23.075861 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:45:23.082260 12771 raft_consensus.cc:1261] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5 [term 1 FOLLOWER]: Refusing update from remote peer 67588ac5c0454bf8b16676e0e4d45851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:23.082300 12919 raft_consensus.cc:1261] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 1 FOLLOWER]: Refusing update from remote peer 67588ac5c0454bf8b16676e0e4d45851: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:23.082944 13004 consensus_queue.cc:1035] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c82afa644524890adfac534ea0448e5" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:23.083146 13000 consensus_queue.cc:1035] T 0f966c36dad245038afad5a1b4a3420e P 67588ac5c0454bf8b16676e0e4d45851 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a11992260104fcda9368d5833bd1d81" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 38503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:23.094866 13009 mvcc.cc:205] Tried to move back new op lower bound from 7316482961736757248 to 7316482961448644608. Current Snapshot: MvccSnapshot[applied={T|T < 7316482961736757248}]
I20260809 04:45:23.101058 12462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:23.101285 12462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:23.104212 12462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de939b011fc8412d8cbb4438358c7a37: Sending DeleteTablet for 3 replicas of tablet 0f966c36dad245038afad5a1b4a3420e
I20260809 04:45:23.105144 12597 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f966c36dad245038afad5a1b4a3420e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:23 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:41570
I20260809 04:45:23.105221 12748 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f966c36dad245038afad5a1b4a3420e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:23 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:58732
I20260809 04:45:23.105209 12899 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f966c36dad245038afad5a1b4a3420e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:23 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:50460
I20260809 04:45:23.105688 13017 tablet_replica.cc:330] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: stopping tablet replica
I20260809 04:45:23.106043 13017 raft_consensus.cc:2229] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:23.106261 13018 tablet_replica.cc:330] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: stopping tablet replica
I20260809 04:45:23.106455 13017 raft_consensus.cc:2258] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:23.106650 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 12531
I20260809 04:45:23.106654 13018 raft_consensus.cc:2229] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:23.108421 13018 raft_consensus.cc:2258] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:23.109932 13018 ts_tablet_manager.cc:1905] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:23.110239 13017 ts_tablet_manager.cc:1905] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:23.112314 13018 ts_tablet_manager.cc:1918] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:23.112432 13018 log.cc:1196] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/0f966c36dad245038afad5a1b4a3420e
I20260809 04:45:23.112793 13018 ts_tablet_manager.cc:1939] T 0f966c36dad245038afad5a1b4a3420e P 2a11992260104fcda9368d5833bd1d81: Deleting consensus metadata
I20260809 04:45:23.114126 13017 ts_tablet_manager.cc:1918] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:23.114257 13017 log.cc:1196] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/0f966c36dad245038afad5a1b4a3420e
I20260809 04:45:23.114315 12444 catalog_manager.cc:4841] TS 2a11992260104fcda9368d5833bd1d81 (127.7.171.3:38503): tablet 0f966c36dad245038afad5a1b4a3420e (table test-table [id=ea4985b315bf45259bd9495ae2c09af5]) successfully deleted
I20260809 04:45:23.114665 13017 ts_tablet_manager.cc:1939] T 0f966c36dad245038afad5a1b4a3420e P 0c82afa644524890adfac534ea0448e5: Deleting consensus metadata
I20260809 04:45:23.115952 12445 catalog_manager.cc:4841] TS 0c82afa644524890adfac534ea0448e5 (127.7.171.2:42393): tablet 0f966c36dad245038afad5a1b4a3420e (table test-table [id=ea4985b315bf45259bd9495ae2c09af5]) successfully deleted
W20260809 04:45:23.119215 12445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:42933 (error 108)
I20260809 04:45:23.120172 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 12679
I20260809 04:45:23.120467 12986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.120053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:42933 (local address 127.0.0.1:41596)
0809 04:45:23.120280 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:23.120356 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:42933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260809 04:45:23.121378 12445 catalog_manager.cc:4584] TS 67588ac5c0454bf8b16676e0e4d45851 (127.7.171.1:42933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f966c36dad245038afad5a1b4a3420e: Network error: Client connection negotiation failed: client connection to 127.7.171.1:42933: connect: Connection refused (error 111)
I20260809 04:45:23.129115 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 12829
I20260809 04:45:23.135943 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 12426
2026-08-09T04:45:23Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2754 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T04:45:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:23Z Disabled control of system clock
I20260809 04:45:23.172998 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:41157
--webserver_interface=127.7.171.62
--webserver_port=0
--builtin_ntp_servers=127.7.171.20:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:41157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 04:45:23.287384 13024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:23.287695 13024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:23.287734 13024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:23.291683 13024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:23.291750 13024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:23.291765 13024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:23.291782 13024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:23.291798 13024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:23.296574 13024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:41157
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:41157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:23.298244 13024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:23.299829 13024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:23.309294 13033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:23.309343 13030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:23.309294 13031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:23.309594 13024 server_base.cc:1034] running on GCE node
I20260809 04:45:23.310371 13024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:23.311602 13024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:23.312825 13024 hybrid_clock.cc:648] HybridClock initialized: now 1786250723312808 us; error 36 us; skew 500 ppm
I20260809 04:45:23.315047 13024 webserver.cc:460] Webserver started at http://127.7.171.62:45893/ using document root <none> and password file <none>
I20260809 04:45:23.315716 13024 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:23.315824 13024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:23.316144 13024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:23.318418 13024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "97b006d7012c45fa91f0eff0cd1c95b7"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "2518ad9ac3466d87d6b771d29a8c58e7"
server_key_iv: "556a53b48ac60a71502e2a0745187400"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.319083 13024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "97b006d7012c45fa91f0eff0cd1c95b7"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "2518ad9ac3466d87d6b771d29a8c58e7"
server_key_iv: "556a53b48ac60a71502e2a0745187400"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.323226 13024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260809 04:45:23.326103 13039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:23.327538 13024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 04:45:23.327701 13024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "97b006d7012c45fa91f0eff0cd1c95b7"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "2518ad9ac3466d87d6b771d29a8c58e7"
server_key_iv: "556a53b48ac60a71502e2a0745187400"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.327831 13024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:23.332633 13024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:23.333278 13024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:23.333472 13024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:23.340782 13024 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:41157
I20260809 04:45:23.340814 13096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:41157 every 8 connection(s)
I20260809 04:45:23.341764 13024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:23.345132 13097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:23.349475 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 13024
I20260809 04:45:23.349628 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:23.349905 7852 external_mini_cluster.cc:1428] Setting key 0f3287b0e96c47adfc9d5bf8b0a672cd
I20260809 04:45:23.351276 13097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7: Bootstrap starting.
I20260809 04:45:23.353714 13097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:23.354583 13097 log.cc:824] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:23.355823 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.351385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.7.171.62:41157)
0809 04:45:23.352102 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:23.352130 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:23.352166 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:45:23.352819 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:23.352836 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:23.353198 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:23.353548 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:23.354342 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:23.354348 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:23.354365 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:23.354381 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:23.354433 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:23.354448 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:23.354571 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:23.354727 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 04:45:23.354975 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":128,"threads_started":1}
I20260809 04:45:23.356775 13097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7: No bootstrap required, opened a new log
I20260809 04:45:23.360632 13097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b006d7012c45fa91f0eff0cd1c95b7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41157 } }
I20260809 04:45:23.360888 13097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:23.360953 13097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b006d7012c45fa91f0eff0cd1c95b7, State: Initialized, Role: FOLLOWER
I20260809 04:45:23.361440 13097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b006d7012c45fa91f0eff0cd1c95b7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41157 } }
I20260809 04:45:23.361604 13097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:23.361691 13097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:23.361812 13097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:23.362819 13097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b006d7012c45fa91f0eff0cd1c95b7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41157 } }
I20260809 04:45:23.363168 13097 leader_election.cc:302] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97b006d7012c45fa91f0eff0cd1c95b7; no voters:
I20260809 04:45:23.363483 13097 leader_election.cc:288] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:23.363622 13102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:23.363888 13102 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [term 1 LEADER]: Becoming Leader. State: Replica: 97b006d7012c45fa91f0eff0cd1c95b7, State: Running, Role: LEADER
I20260809 04:45:23.364284 13102 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b006d7012c45fa91f0eff0cd1c95b7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41157 } }
I20260809 04:45:23.364856 13097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:23.366060 13104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97b006d7012c45fa91f0eff0cd1c95b7. Latest consensus state: current_term: 1 leader_uuid: "97b006d7012c45fa91f0eff0cd1c95b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b006d7012c45fa91f0eff0cd1c95b7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41157 } } }
I20260809 04:45:23.366060 13103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97b006d7012c45fa91f0eff0cd1c95b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b006d7012c45fa91f0eff0cd1c95b7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41157 } } }
I20260809 04:45:23.366225 13104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:23.366225 13103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:23.366707 13111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:23.369889 13111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:23.375873 13111 catalog_manager.cc:1347] Generated new cluster ID: 9e9fe06d407049b8bb339121a32d9521
I20260809 04:45:23.375952 13111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:23.383529 13111 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:23.383843 13111 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:23.393927 13111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7: Generated new TSK 0
I20260809 04:45:23.394646 13111 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:45:23.403395 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41157
--builtin_ntp_servers=127.7.171.20:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 04:45:23.528652 13122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:23.528914 13122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:23.528956 13122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:23.533198 13122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:23.533263 13122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:23.533349 13122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:23.538226 13122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:23.539439 13122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:23.540928 13122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:23.551272 13132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:23.551379 13122 server_base.cc:1034] running on GCE node
W20260809 04:45:23.551272 13128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:23.551267 13129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:23.552217 13122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:23.552950 13122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:23.554175 13122 hybrid_clock.cc:648] HybridClock initialized: now 1786250723554161 us; error 26 us; skew 500 ppm
I20260809 04:45:23.556439 13122 webserver.cc:460] Webserver started at http://127.7.171.1:36307/ using document root <none> and password file <none>
I20260809 04:45:23.557040 13122 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:23.557147 13122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:23.557395 13122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:23.559563 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "3d8baebbb27b43fda40b7a309d0c6a8e"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "b6d59d549b3656cf04af3c8a61f89613"
server_key_iv: "d86f85d61171b42d65f5e5896870fe8a"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.560231 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "3d8baebbb27b43fda40b7a309d0c6a8e"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "b6d59d549b3656cf04af3c8a61f89613"
server_key_iv: "d86f85d61171b42d65f5e5896870fe8a"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.564138 13122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:23.566970 13138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:23.568382 13122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:23.568517 13122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "3d8baebbb27b43fda40b7a309d0c6a8e"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "b6d59d549b3656cf04af3c8a61f89613"
server_key_iv: "d86f85d61171b42d65f5e5896870fe8a"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.568634 13122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:23.573432 13122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:23.574045 13122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:23.574244 13122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:23.574810 13122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:23.575747 13122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:23.575824 13122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:23.575902 13122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:23.575945 13122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:23.585660 13122 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:39297
I20260809 04:45:23.585734 13263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:39297 every 8 connection(s)
I20260809 04:45:23.586655 13122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:23.592077 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 13122
I20260809 04:45:23.592224 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:23.592655 7852 external_mini_cluster.cc:1428] Setting key 9cffb77eb11c7ce52e8516a04bd2bc39
I20260809 04:45:23.593415 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.588637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:36713 (local address 127.7.171.62:41157)
0809 04:45:23.588813 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:23.588818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:23.589964 (+ 1146us) server_negotiation.cc:407] Connection header received
0809 04:45:23.591311 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:23.591314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:23.591354 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:23.591432 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:23.592906 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:23.592909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:23.592911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:23.592924 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:23.592944 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:23.592953 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:23.593033 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:23.593226 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 04:45:23.593276 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:45:23.594457 13267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.589076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:41157 (local address 127.7.171.1:36713)
0809 04:45:23.589793 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:23.589841 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:23.591115 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:23.591641 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:23.591653 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:23.591788 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:23.592577 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:23.592581 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:23.592639 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:23.592734 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:23.593044 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:23.593056 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:23.593065 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:45:23.593228 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 04:45:23.593487 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":128,"threads_started":1}
I20260809 04:45:23.594913 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41157
--builtin_ntp_servers=127.7.171.20:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 04:45:23.595736 13264 heartbeater.cc:347] Connected to a master server at 127.7.171.62:41157
I20260809 04:45:23.595985 13264 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:23.596442 13264 heartbeater.cc:511] Master 127.7.171.62:41157 requested a full tablet report, sending...
I20260809 04:45:23.598161 13058 ts_manager.cc:194] Registered new tserver with Master: 3d8baebbb27b43fda40b7a309d0c6a8e (127.7.171.1:39297)
W20260809 04:45:23.708163 13268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:23.708431 13268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:23.708472 13268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:23.712533 13268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:23.712596 13268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:23.712656 13268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:23.717468 13268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:23.718679 13268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:23.720166 13268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:23.729948 13274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:23.729956 13276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:23.730064 13278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:23.730315 13268 server_base.cc:1034] running on GCE node
I20260809 04:45:23.730739 13268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:23.731427 13268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:23.732633 13268 hybrid_clock.cc:648] HybridClock initialized: now 1786250723732620 us; error 27 us; skew 500 ppm
I20260809 04:45:23.734506 13268 webserver.cc:460] Webserver started at http://127.7.171.2:35751/ using document root <none> and password file <none>
I20260809 04:45:23.735116 13268 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:23.735184 13268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:23.735409 13268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:23.737551 13268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "000b750d8e6643e4af5c8e36fc907317"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "7f62fe3739b67ff7044f44ac53a18b4c"
server_key_iv: "42ee6f0edcdfeb7745a2563dced406f2"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.738138 13268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "000b750d8e6643e4af5c8e36fc907317"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "7f62fe3739b67ff7044f44ac53a18b4c"
server_key_iv: "42ee6f0edcdfeb7745a2563dced406f2"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.742041 13268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:45:23.744599 13284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:23.745823 13268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:23.745939 13268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "000b750d8e6643e4af5c8e36fc907317"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "7f62fe3739b67ff7044f44ac53a18b4c"
server_key_iv: "42ee6f0edcdfeb7745a2563dced406f2"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.746074 13268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:23.751344 13268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:23.751956 13268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:23.752173 13268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:23.752748 13268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:23.753675 13268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:23.753722 13268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:23.753793 13268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:23.753826 13268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:23.762997 13268 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:36581
I20260809 04:45:23.763023 13416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:36581 every 8 connection(s)
I20260809 04:45:23.763934 13268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:23.770413 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.766038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:34537 (local address 127.7.171.62:41157)
0809 04:45:23.766197 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:23.766201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:23.767352 (+ 1151us) server_negotiation.cc:407] Connection header received
0809 04:45:23.768738 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:23.768741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:23.768779 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:23.768856 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:23.770010 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:23.770013 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:23.770015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:23.770026 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:23.770047 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:23.770055 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:23.770143 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:23.770263 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 04:45:23.770301 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:45:23.771341 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.766487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:41157 (local address 127.7.171.2:34537)
0809 04:45:23.767208 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:23.767248 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:23.768525 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:23.769027 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:23.769034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:23.769128 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:23.769740 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:23.769742 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:23.769784 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:23.769854 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:23.770142 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:23.770149 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:23.770153 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:45:23.770285 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 04:45:23.770549 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":141,"threads_started":1}
I20260809 04:45:23.772727 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 13268
I20260809 04:45:23.772763 13417 heartbeater.cc:347] Connected to a master server at 127.7.171.62:41157
I20260809 04:45:23.772855 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:23.773038 13417 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:23.773223 7852 external_mini_cluster.cc:1428] Setting key 5548d41d139c55dd2e656e86798ba166
I20260809 04:45:23.773523 13417 heartbeater.cc:511] Master 127.7.171.62:41157 requested a full tablet report, sending...
I20260809 04:45:23.774616 13057 ts_manager.cc:194] Registered new tserver with Master: 000b750d8e6643e4af5c8e36fc907317 (127.7.171.2:36581)
I20260809 04:45:23.775358 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41157
--builtin_ntp_servers=127.7.171.20:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 04:45:23.899626 13421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:23.899907 13421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:23.899986 13421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:23.904122 13421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:23.904189 13421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:23.904284 13421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:23.909221 13421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:23.910418 13421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:23.911904 13421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:23.922501 13428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:23.922535 13427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:23.922540 13421 server_base.cc:1034] running on GCE node
W20260809 04:45:23.922499 13430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:23.923164 13421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:23.923800 13421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:23.924964 13421 hybrid_clock.cc:648] HybridClock initialized: now 1786250723924928 us; error 48 us; skew 500 ppm
I20260809 04:45:23.927014 13421 webserver.cc:460] Webserver started at http://127.7.171.3:44283/ using document root <none> and password file <none>
I20260809 04:45:23.927624 13421 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:23.927690 13421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:23.927920 13421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:23.930120 13421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "0f34034257344442bbdad61d89863add"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "d5ed1af67361f1b0f98ea49626886a13"
server_key_iv: "455b4ffa2a7bb5e179ece8e92dd11739"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.930729 13421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "0f34034257344442bbdad61d89863add"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "d5ed1af67361f1b0f98ea49626886a13"
server_key_iv: "455b4ffa2a7bb5e179ece8e92dd11739"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.934729 13421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 04:45:23.937273 13436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:23.938524 13421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:45:23.938685 13421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "0f34034257344442bbdad61d89863add"
format_stamp: "Formatted at 2026-08-09 04:45:23 on dist-test-slave-1zqn"
server_key: "d5ed1af67361f1b0f98ea49626886a13"
server_key_iv: "455b4ffa2a7bb5e179ece8e92dd11739"
server_key_version: "encryptionkey@0"
I20260809 04:45:23.938807 13421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:23.943665 13421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:23.944368 13421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:23.944592 13421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:23.945165 13421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:23.946116 13421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:23.946189 13421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:23.946264 13421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:23.946307 13421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:23.955837 13421 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:45213
I20260809 04:45:23.955860 13567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:45213 every 8 connection(s)
I20260809 04:45:23.956813 13421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:23.962776 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 13421
I20260809 04:45:23.962905 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:23.963286 7852 external_mini_cluster.cc:1428] Setting key ffc730dc594bdb9ad3a48ebc0ca24039
I20260809 04:45:23.963536 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.958769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:60139 (local address 127.7.171.62:41157)
0809 04:45:23.958928 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:23.958932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:23.960203 (+ 1271us) server_negotiation.cc:407] Connection header received
0809 04:45:23.961557 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:23.961560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:23.961605 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:23.961697 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:23.962904 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:23.962908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:23.962910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:23.962923 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:23.962946 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:23.962958 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:23.963076 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:23.963289 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 04:45:23.963354 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:45:23.964643 13570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.959200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:41157 (local address 127.7.171.3:60139)
0809 04:45:23.959925 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:23.960031 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:23.961343 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:23.961881 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:23.961888 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:23.961982 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:23.962638 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:23.962640 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:23.962681 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:23.962751 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:23.963068 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:23.963080 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:23.963087 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 04:45:23.963290 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 04:45:23.963545 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":126,"threads_started":1}
I20260809 04:45:23.965742 13568 heartbeater.cc:347] Connected to a master server at 127.7.171.62:41157
I20260809 04:45:23.965946 13568 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:23.966396 13568 heartbeater.cc:511] Master 127.7.171.62:41157 requested a full tablet report, sending...
I20260809 04:45:23.967610 13057 ts_manager.cc:194] Registered new tserver with Master: 0f34034257344442bbdad61d89863add (127.7.171.3:45213)
I20260809 04:45:23.977691 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:45:23.981474 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.979805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.7.171.62:41157)
0809 04:45:23.979978 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:23.979984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:23.980204 (+ 220us) server_negotiation.cc:407] Connection header received
0809 04:45:23.980315 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:23.980317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:23.980354 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:23.980416 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:23.981089 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:23.981092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:23.981094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:23.981104 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:23.981122 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:23.981130 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:23.981198 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:23.981270 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 04:45:23.981301 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:45:23.984434 13057 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:23.988019 13057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:24.000653 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.997297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:36581 (local address 127.0.0.1:58624)
0809 04:45:23.997935 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:23.997992 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:23.998124 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:23.998996 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:23.999002 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:23.999090 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:23.999801 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:23.999803 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:23.999825 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:23.999898 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:24.000272 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:24.000281 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:24.000286 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:45:24.000421 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 04:45:24.000494 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"spinlock_wait_cycles":4480,"thread_start_us":120,"threads_started":1}
I20260809 04:45:24.000690 13585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.997466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58624 (local address 127.7.171.2:36581)
0809 04:45:23.998413 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:23.998431 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:23.998458 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:45:23.998518 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:23.998524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:23.998809 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:23.999410 (+ 601us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:24.000089 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:24.000095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:24.000109 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:24.000122 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:24.000166 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:24.000201 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:24.000304 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:24.000449 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 04:45:24.000526 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":142,"threads_started":1}
I20260809 04:45:24.001318 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.997672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:39297 (local address 127.0.0.1:41202)
0809 04:45:23.997935 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:23.997992 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:23.998124 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:23.999171 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:23.999177 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:23.999193 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:24.000072 (+ 879us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:24.000075 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:24.000094 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:24.000118 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:24.000699 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:24.000702 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:24.000706 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:45:24.001130 (+ 424us) client_negotiation.cc:240] Negotiation successful
0809 04:45:24.001167 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"spinlock_wait_cycles":40832,"thread_start_us":90,"threads_started":1}
I20260809 04:45:24.001847 13590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.998470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.7.171.3:45213)
0809 04:45:23.998990 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:23.999002 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:23.999028 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:45:23.999082 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:23.999090 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:23.999249 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:23.999926 (+ 677us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:24.001110 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:24.001115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:24.001125 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:24.001139 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:24.001181 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:24.001213 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:24.001310 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:24.001612 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 04:45:24.001693 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":164,"threads_started":1}
I20260809 04:45:24.002032 13589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.998152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41202 (local address 127.7.171.1:39297)
0809 04:45:23.998709 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:23.998722 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:23.998748 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:45:23.998805 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:23.998813 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:23.998989 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:23.999181 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:24.000259 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:24.000269 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:24.000283 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:24.000298 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:24.000347 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:24.000388 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:24.001734 (+ 1346us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:24.001798 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 04:45:24.001869 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":150,"threads_started":1}
I20260809 04:45:24.002144 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:23.998079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45213 (local address 127.0.0.1:34710)
0809 04:45:23.998924 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:23.998939 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:23.999004 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:23.999918 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:23.999921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:23.999938 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:24.000533 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:24.000537 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:24.000551 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:24.000584 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:24.001353 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:24.001356 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:24.001362 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:45:24.001442 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:45:24.001478 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":91,"threads_started":1}
I20260809 04:45:24.003825 13344 tablet_service.cc:1467] Processing CreateTablet for tablet 6cb03514a6da42d1a0e5a183666a19c4 (DEFAULT_TABLE table=test-table [id=5b2e41e8b5f14e50a8648824f565374d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:24.005057 13344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cb03514a6da42d1a0e5a183666a19c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:24.004851 13192 tablet_service.cc:1467] Processing CreateTablet for tablet 6cb03514a6da42d1a0e5a183666a19c4 (DEFAULT_TABLE table=test-table [id=5b2e41e8b5f14e50a8648824f565374d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:24.004848 13494 tablet_service.cc:1467] Processing CreateTablet for tablet 6cb03514a6da42d1a0e5a183666a19c4 (DEFAULT_TABLE table=test-table [id=5b2e41e8b5f14e50a8648824f565374d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:24.005882 13192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cb03514a6da42d1a0e5a183666a19c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:24.005877 13494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cb03514a6da42d1a0e5a183666a19c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:24.011407 13591 tablet_bootstrap.cc:492] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317: Bootstrap starting.
I20260809 04:45:24.013700 13593 tablet_bootstrap.cc:492] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add: Bootstrap starting.
I20260809 04:45:24.013808 13592 tablet_bootstrap.cc:492] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e: Bootstrap starting.
I20260809 04:45:24.014034 13591 tablet_bootstrap.cc:654] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:24.014781 13591 log.cc:824] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:24.015942 13593 tablet_bootstrap.cc:654] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:24.015940 13592 tablet_bootstrap.cc:654] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:24.016724 13591 tablet_bootstrap.cc:492] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317: No bootstrap required, opened a new log
I20260809 04:45:24.016825 13592 log.cc:824] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:24.016825 13593 log.cc:824] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:24.016956 13591 ts_tablet_manager.cc:1397] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:45:24.018875 13593 tablet_bootstrap.cc:492] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add: No bootstrap required, opened a new log
I20260809 04:45:24.018875 13592 tablet_bootstrap.cc:492] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e: No bootstrap required, opened a new log
I20260809 04:45:24.019068 13593 ts_tablet_manager.cc:1397] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:45:24.019120 13592 ts_tablet_manager.cc:1397] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:45:24.021076 13591 raft_consensus.cc:348] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000b750d8e6643e4af5c8e36fc907317" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36581 } } peers { permanent_uuid: "0f34034257344442bbdad61d89863add" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45213 } } peers { permanent_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 39297 } }
I20260809 04:45:24.021329 13591 raft_consensus.cc:374] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:24.021389 13591 raft_consensus.cc:729] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 000b750d8e6643e4af5c8e36fc907317, State: Initialized, Role: FOLLOWER
I20260809 04:45:24.021834 13591 consensus_queue.cc:260] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000b750d8e6643e4af5c8e36fc907317" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36581 } } peers { permanent_uuid: "0f34034257344442bbdad61d89863add" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45213 } } peers { permanent_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 39297 } }
I20260809 04:45:24.022492 13417 heartbeater.cc:503] Master 127.7.171.62:41157 was elected leader, sending a full tablet report...
I20260809 04:45:24.023620 13593 raft_consensus.cc:348] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000b750d8e6643e4af5c8e36fc907317" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36581 } } peers { permanent_uuid: "0f34034257344442bbdad61d89863add" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45213 } } peers { permanent_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 39297 } }
I20260809 04:45:24.023620 13592 raft_consensus.cc:348] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000b750d8e6643e4af5c8e36fc907317" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36581 } } peers { permanent_uuid: "0f34034257344442bbdad61d89863add" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45213 } } peers { permanent_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 39297 } }
I20260809 04:45:24.023846 13591 ts_tablet_manager.cc:1428] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 04:45:24.023874 13593 raft_consensus.cc:374] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:24.023874 13592 raft_consensus.cc:374] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:24.023926 13593 raft_consensus.cc:729] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f34034257344442bbdad61d89863add, State: Initialized, Role: FOLLOWER
I20260809 04:45:24.023936 13592 raft_consensus.cc:729] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d8baebbb27b43fda40b7a309d0c6a8e, State: Initialized, Role: FOLLOWER
I20260809 04:45:24.024394 13592 consensus_queue.cc:260] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000b750d8e6643e4af5c8e36fc907317" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36581 } } peers { permanent_uuid: "0f34034257344442bbdad61d89863add" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45213 } } peers { permanent_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 39297 } }
I20260809 04:45:24.024511 13593 consensus_queue.cc:260] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000b750d8e6643e4af5c8e36fc907317" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36581 } } peers { permanent_uuid: "0f34034257344442bbdad61d89863add" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45213 } } peers { permanent_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 39297 } }
I20260809 04:45:24.025156 13264 heartbeater.cc:503] Master 127.7.171.62:41157 was elected leader, sending a full tablet report...
I20260809 04:45:24.025468 13592 ts_tablet_manager.cc:1428] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 04:45:24.025468 13593 ts_tablet_manager.cc:1428] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260809 04:45:24.025799 13568 heartbeater.cc:503] Master 127.7.171.62:41157 was elected leader, sending a full tablet report...
I20260809 04:45:24.042707 13600 raft_consensus.cc:482] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:24.042975 13600 raft_consensus.cc:504] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000b750d8e6643e4af5c8e36fc907317" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36581 } } peers { permanent_uuid: "0f34034257344442bbdad61d89863add" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45213 } } peers { permanent_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 39297 } }
I20260809 04:45:24.044260 13600 leader_election.cc:288] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 000b750d8e6643e4af5c8e36fc907317 (127.7.171.2:36581), 0f34034257344442bbdad61d89863add (127.7.171.3:45213)
I20260809 04:45:24.046389 13267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:24.044541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:36581 (local address 127.7.171.1:38777)
0809 04:45:24.044684 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:24.044702 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:24.044786 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:24.045129 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:24.045133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:24.045149 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:24.045719 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:24.045721 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:24.045734 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:24.045764 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:24.046125 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:24.046127 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:24.046130 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 04:45:24.046191 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 04:45:24.046248 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 04:45:24.046365 13585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:24.044664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:38777 (local address 127.7.171.2:36581)
0809 04:45:24.044795 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:24.044800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:24.044854 (+ 54us) server_negotiation.cc:407] Connection header received
0809 04:45:24.044906 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:24.044909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:24.044949 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:24.045041 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:24.045961 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:24.045965 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:24.045968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:24.045986 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:24.046013 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:24.046024 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:24.046138 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:24.046204 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 04:45:24.046250 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:45:24.047057 13368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cb03514a6da42d1a0e5a183666a19c4" candidate_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000b750d8e6643e4af5c8e36fc907317" is_pre_election: true
I20260809 04:45:24.047266 13590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:24.045173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:40981 (local address 127.7.171.3:45213)
0809 04:45:24.045336 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:24.045343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:24.045622 (+ 279us) server_negotiation.cc:407] Connection header received
0809 04:45:24.045734 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:24.045737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:24.045775 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:24.045853 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:24.046871 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:24.046875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:24.046877 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:24.046890 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:24.046912 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:24.046921 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:24.047020 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:24.047111 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 04:45:24.047151 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:45:24.047318 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:24.044836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45213 (local address 127.7.171.1:40981)
0809 04:45:24.045541 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:24.045556 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:24.045647 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:24.046018 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:24.046021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:24.046038 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:24.046734 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:24.046736 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:24.046744 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:24.046767 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:24.047025 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:24.047027 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:24.047030 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 04:45:24.047113 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 04:45:24.047140 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":137,"threads_started":1}
I20260809 04:45:24.047382 13368 raft_consensus.cc:2454] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d8baebbb27b43fda40b7a309d0c6a8e in term 0.
I20260809 04:45:24.047909 13139 leader_election.cc:302] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000b750d8e6643e4af5c8e36fc907317, 3d8baebbb27b43fda40b7a309d0c6a8e; no voters:
I20260809 04:45:24.048244 13600 raft_consensus.cc:2781] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:24.048275 13515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cb03514a6da42d1a0e5a183666a19c4" candidate_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f34034257344442bbdad61d89863add" is_pre_election: true
I20260809 04:45:24.048358 13600 raft_consensus.cc:482] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:24.048408 13600 raft_consensus.cc:3037] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:24.048549 13515 raft_consensus.cc:2454] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d8baebbb27b43fda40b7a309d0c6a8e in term 0.
I20260809 04:45:24.049549 13600 raft_consensus.cc:504] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000b750d8e6643e4af5c8e36fc907317" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36581 } } peers { permanent_uuid: "0f34034257344442bbdad61d89863add" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45213 } } peers { permanent_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 39297 } }
I20260809 04:45:24.049830 13600 leader_election.cc:288] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [CANDIDATE]: Term 1 election: Requested vote from peers 000b750d8e6643e4af5c8e36fc907317 (127.7.171.2:36581), 0f34034257344442bbdad61d89863add (127.7.171.3:45213)
I20260809 04:45:24.050200 13368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cb03514a6da42d1a0e5a183666a19c4" candidate_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "000b750d8e6643e4af5c8e36fc907317"
I20260809 04:45:24.050338 13368 raft_consensus.cc:3037] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:24.050365 13515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cb03514a6da42d1a0e5a183666a19c4" candidate_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f34034257344442bbdad61d89863add"
I20260809 04:45:24.050514 13515 raft_consensus.cc:3037] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:24.051644 13368 raft_consensus.cc:2454] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d8baebbb27b43fda40b7a309d0c6a8e in term 1.
I20260809 04:45:24.051690 13515 raft_consensus.cc:2454] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d8baebbb27b43fda40b7a309d0c6a8e in term 1.
I20260809 04:45:24.051937 13139 leader_election.cc:302] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 000b750d8e6643e4af5c8e36fc907317, 3d8baebbb27b43fda40b7a309d0c6a8e; no voters:
I20260809 04:45:24.052270 13600 raft_consensus.cc:2781] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:24.052546 13600 raft_consensus.cc:686] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [term 1 LEADER]: Becoming Leader. State: Replica: 3d8baebbb27b43fda40b7a309d0c6a8e, State: Running, Role: LEADER
I20260809 04:45:24.052909 13600 consensus_queue.cc:237] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000b750d8e6643e4af5c8e36fc907317" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36581 } } peers { permanent_uuid: "0f34034257344442bbdad61d89863add" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45213 } } peers { permanent_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 39297 } }
I20260809 04:45:24.055986 13056 catalog_manager.cc:5495] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d8baebbb27b43fda40b7a309d0c6a8e (127.7.171.1). New cstate: current_term: 1 leader_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "000b750d8e6643e4af5c8e36fc907317" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34034257344442bbdad61d89863add" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d8baebbb27b43fda40b7a309d0c6a8e" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 39297 } health_report { overall_health: HEALTHY } } }
I20260809 04:45:24.081292 13589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:24.079386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41218 (local address 127.7.171.1:39297)
0809 04:45:24.079560 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:24.079563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:24.079577 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:24.079683 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:24.079685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:24.079762 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:24.079833 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:24.080652 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:24.080655 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:24.080657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:24.080669 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:24.080689 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:24.080698 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:24.080786 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:24.080885 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 04:45:24.081112 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:45:24.087743 13515 raft_consensus.cc:1261] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add [term 1 FOLLOWER]: Refusing update from remote peer 3d8baebbb27b43fda40b7a309d0c6a8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:24.087743 13368 raft_consensus.cc:1261] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317 [term 1 FOLLOWER]: Refusing update from remote peer 3d8baebbb27b43fda40b7a309d0c6a8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 04:45:24.087880 13266 tablet.cc:2369] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:45:24.088625 13600 consensus_queue.cc:1035] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "000b750d8e6643e4af5c8e36fc907317" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:24.088915 13604 consensus_queue.cc:1035] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f34034257344442bbdad61d89863add" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:24.106034 13058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:24.106225 13058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:24.108672 13058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97b006d7012c45fa91f0eff0cd1c95b7: Sending DeleteTablet for 3 replicas of tablet 6cb03514a6da42d1a0e5a183666a19c4
I20260809 04:45:24.109580 13192 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cb03514a6da42d1a0e5a183666a19c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:24 UTC) from {username='slave'} at 127.0.0.1:41202
I20260809 04:45:24.109675 13344 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cb03514a6da42d1a0e5a183666a19c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:24 UTC) from {username='slave'} at 127.0.0.1:58624
I20260809 04:45:24.110138 13494 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cb03514a6da42d1a0e5a183666a19c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:24 UTC) from {username='slave'} at 127.0.0.1:34710
I20260809 04:45:24.110357 13621 tablet_replica.cc:330] T 6cb03514a6da42d1a0e5a183666a19c4 P 3d8baebbb27b43fda40b7a309d0c6a8e: stopping tablet replica
I20260809 04:45:24.110622 13622 tablet_replica.cc:330] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add: stopping tablet replica
I20260809 04:45:24.110774 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 13122
I20260809 04:45:24.110971 13622 raft_consensus.cc:2229] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:24.111299 13622 raft_consensus.cc:2258] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:24.113601 13620 tablet_replica.cc:330] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317: stopping tablet replica
I20260809 04:45:24.113910 13620 raft_consensus.cc:2229] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:24.114228 13620 raft_consensus.cc:2258] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:24.115888 13620 ts_tablet_manager.cc:1905] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:24.117324 13622 ts_tablet_manager.cc:1905] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:24.118839 13620 ts_tablet_manager.cc:1918] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:24.118952 13620 log.cc:1196] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/6cb03514a6da42d1a0e5a183666a19c4
I20260809 04:45:24.119292 13620 ts_tablet_manager.cc:1939] T 6cb03514a6da42d1a0e5a183666a19c4 P 000b750d8e6643e4af5c8e36fc907317: Deleting consensus metadata
I20260809 04:45:24.120203 13622 ts_tablet_manager.cc:1918] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:24.120306 13622 log.cc:1196] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/6cb03514a6da42d1a0e5a183666a19c4
I20260809 04:45:24.120651 13622 ts_tablet_manager.cc:1939] T 6cb03514a6da42d1a0e5a183666a19c4 P 0f34034257344442bbdad61d89863add: Deleting consensus metadata
I20260809 04:45:24.120792 13040 catalog_manager.cc:4841] TS 000b750d8e6643e4af5c8e36fc907317 (127.7.171.2:36581): tablet 6cb03514a6da42d1a0e5a183666a19c4 (table test-table [id=5b2e41e8b5f14e50a8648824f565374d]) successfully deleted
I20260809 04:45:24.121766 13040 catalog_manager.cc:4841] TS 0f34034257344442bbdad61d89863add (127.7.171.3:45213): tablet 6cb03514a6da42d1a0e5a183666a19c4 (table test-table [id=5b2e41e8b5f14e50a8648824f565374d]) successfully deleted
W20260809 04:45:24.122542 13043 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:39297 (error 108)
I20260809 04:45:24.122730 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 13268
I20260809 04:45:24.128235 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:24.127804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:39297 (local address 127.0.0.1:41232)
0809 04:45:24.127994 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:24.128116 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:39297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 04:45:24.128461 13043 catalog_manager.cc:4584] TS 3d8baebbb27b43fda40b7a309d0c6a8e (127.7.171.1:39297): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6cb03514a6da42d1a0e5a183666a19c4: Network error: Client connection negotiation failed: client connection to 127.7.171.1:39297: connect: Connection refused (error 111)
I20260809 04:45:24.131361 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 13421
I20260809 04:45:24.139267 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 13024
2026-08-09T04:45:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1001 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 04:45:24.165031 7852 mini_oidc.cc:152] Starting JWKS server
I20260809 04:45:24.166742 7852 webserver.cc:460] Webserver started at https://127.0.0.1:41923/ using document root <none> and password file <none>
I20260809 04:45:24.166934 7852 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:45:24.167205 7852 webserver.cc:460] Webserver started at http://127.0.0.1:41245/ using document root <none> and password file <none>
2026-08-09T04:45:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:24Z Disabled control of system clock
I20260809 04:45:24.177923 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35445
--webserver_interface=127.7.171.62
--webserver_port=0
--builtin_ntp_servers=127.7.171.20:46265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:35445
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41923/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/testchainca.cert with env {}
W20260809 04:45:24.294402 13631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:24.294696 13631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:24.294739 13631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:24.298789 13631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:24.298861 13631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:24.298875 13631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:24.298893 13631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:24.298909 13631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:45:24.298925 13631 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:45:24.298939 13631 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41923/jwks.json
I20260809 04:45:24.304467 13631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:46265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:35445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41923/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:24.305495 13631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:24.306483 13631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:24.315153 13637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:24.315158 13640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:24.315352 13631 server_base.cc:1034] running on GCE node
W20260809 04:45:24.315443 13636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:24.316191 13631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:24.317348 13631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:24.318524 13631 hybrid_clock.cc:648] HybridClock initialized: now 1786250724318497 us; error 39 us; skew 500 ppm
I20260809 04:45:24.320771 13631 webserver.cc:460] Webserver started at http://127.7.171.62:45635/ using document root <none> and password file <none>
I20260809 04:45:24.321417 13631 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:24.321521 13631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:24.321766 13631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:24.324019 13631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "6ceb6238e35f4bc791b3d92a9f1bbf0c"
format_stamp: "Formatted at 2026-08-09 04:45:24 on dist-test-slave-1zqn"
server_key: "b9a75150d7caf438223dc5976dad6478"
server_key_iv: "e4247dec2f9e0eee711a90ad17ff581c"
server_key_version: "encryptionkey@0"
I20260809 04:45:24.324723 13631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "6ceb6238e35f4bc791b3d92a9f1bbf0c"
format_stamp: "Formatted at 2026-08-09 04:45:24 on dist-test-slave-1zqn"
server_key: "b9a75150d7caf438223dc5976dad6478"
server_key_iv: "e4247dec2f9e0eee711a90ad17ff581c"
server_key_version: "encryptionkey@0"
I20260809 04:45:24.328691 13631 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:24.331226 13647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:24.332674 13631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:24.332834 13631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "6ceb6238e35f4bc791b3d92a9f1bbf0c"
format_stamp: "Formatted at 2026-08-09 04:45:24 on dist-test-slave-1zqn"
server_key: "b9a75150d7caf438223dc5976dad6478"
server_key_iv: "e4247dec2f9e0eee711a90ad17ff581c"
server_key_version: "encryptionkey@0"
I20260809 04:45:24.332978 13631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:24.349653 13631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:24.350402 13631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:24.350625 13631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:24.358142 13631 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:35445
I20260809 04:45:24.358176 13710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:35445 every 8 connection(s)
I20260809 04:45:24.359241 13631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:24.362751 13713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:24.364971 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 13631
I20260809 04:45:24.365115 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:24.365406 7852 external_mini_cluster.cc:1428] Setting key 938d7b7afde0de120817efbd47874e52
I20260809 04:45:24.368935 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c: Bootstrap starting.
I20260809 04:45:24.371182 13713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:24.371886 13713 log.cc:824] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:24.373793 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c: No bootstrap required, opened a new log
I20260809 04:45:24.375007 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:24.366838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.7.171.62:35445)
0809 04:45:24.367563 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:24.367580 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:24.367618 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:45:24.368415 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:24.368437 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:24.368835 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:24.369316 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:24.369788 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.370735 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.371708 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.372281 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.372468 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:24.373331 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:24.373336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:24.373349 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:24.373738 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:24.373791 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:24.373811 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:24.373921 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:24.374069 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 04:45:24.374320 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":139,"threads_started":1}
I20260809 04:45:24.378111 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:45:24.378036 13713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ceb6238e35f4bc791b3d92a9f1bbf0c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35445 } }
I20260809 04:45:24.378333 13713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:24.378401 13713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ceb6238e35f4bc791b3d92a9f1bbf0c, State: Initialized, Role: FOLLOWER
I20260809 04:45:24.378863 13713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ceb6238e35f4bc791b3d92a9f1bbf0c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35445 } }
I20260809 04:45:24.379024 13713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:24.379096 13713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:24.379194 13713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:24.380357 13713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ceb6238e35f4bc791b3d92a9f1bbf0c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35445 } }
I20260809 04:45:24.380692 13713 leader_election.cc:302] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ceb6238e35f4bc791b3d92a9f1bbf0c; no voters:
I20260809 04:45:24.381024 13713 leader_election.cc:288] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:24.381181 13719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:24.381457 13719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [term 1 LEADER]: Becoming Leader. State: Replica: 6ceb6238e35f4bc791b3d92a9f1bbf0c, State: Running, Role: LEADER
I20260809 04:45:24.381783 13719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ceb6238e35f4bc791b3d92a9f1bbf0c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35445 } }
I20260809 04:45:24.382244 13713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:24.383616 13720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ceb6238e35f4bc791b3d92a9f1bbf0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ceb6238e35f4bc791b3d92a9f1bbf0c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35445 } } }
I20260809 04:45:24.383786 13720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:24.384214 13721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ceb6238e35f4bc791b3d92a9f1bbf0c. Latest consensus state: current_term: 1 leader_uuid: "6ceb6238e35f4bc791b3d92a9f1bbf0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ceb6238e35f4bc791b3d92a9f1bbf0c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35445 } } }
I20260809 04:45:24.384341 13721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c [sys.catalog]: This master's current role is: LEADER
W20260809 04:45:24.387380 13735 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:45:24.387466 13735 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:45:24.387538 13731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:24.388291 13731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:24.393817 13731 catalog_manager.cc:1347] Generated new cluster ID: b5b0ec71c741405c89550b97094cbd61
I20260809 04:45:24.393891 13731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:24.409046 13731 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:24.409987 13731 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:24.415684 13731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ceb6238e35f4bc791b3d92a9f1bbf0c: Generated new TSK 0
I20260809 04:45:24.416560 13731 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 04:45:24.457357 13745 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 04:45:24.457584 13745 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 04:45:24.458355 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:24.434378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55510 (local address 127.7.171.62:35445)
0809 04:45:24.434551 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:24.434556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:24.434790 (+ 234us) server_negotiation.cc:407] Connection header received
0809 04:45:24.435030 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:24.435033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:24.435089 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:24.435193 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:45:24.435515 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.436171 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.437465 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.437763 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.437907 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:24.438313 (+ 406us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:45:24.457777 (+ 19464us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41923/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 04:45:24.458176 (+ 399us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55510: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41923/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 04:45:24.458438 13716 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55510: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41923/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 04:45:24.459745 13744 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:24.434296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35445 (local address 127.0.0.1:55510)
0809 04:45:24.434671 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:24.434688 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:24.434827 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:24.435230 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:24.435233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:24.435241 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:45:24.435371 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:24.435378 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.436290 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.436294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:24.437329 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:24.437340 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.437963 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.437969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:24.438061 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:24.438105 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:45:24.458137 (+ 20032us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41923/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 04:45:24.458399 (+ 262us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.171.62:35445: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41923/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":137,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.171.62:35445: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41923/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 04:45:24.460717 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 13631
2026-08-09T04:45:24Z chronyd exiting
I20260809 04:45:24.501261 7852 test_util.cc:175] -----------------------------------------------
I20260809 04:45:24.501353 7852 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250697618759-7852-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (338 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 04:45:24.502568 7852 mini_oidc.cc:152] Starting JWKS server
I20260809 04:45:24.503451 7852 webserver.cc:460] Webserver started at https://127.0.0.1:39021/ using document root <none> and password file <none>
I20260809 04:45:24.503569 7852 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:45:24.503877 7852 webserver.cc:460] Webserver started at http://127.0.0.1:37079/ using document root <none> and password file <none>
2026-08-09T04:45:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:24Z Disabled control of system clock
I20260809 04:45:24.514339 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46227
--webserver_interface=127.7.171.62
--webserver_port=0
--builtin_ntp_servers=127.7.171.20:38305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:46227
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39021/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/testchainca.cert with env {}
W20260809 04:45:24.632988 13752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:24.633276 13752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:24.633319 13752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:24.637373 13752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:24.637446 13752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:24.637465 13752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:24.637486 13752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:24.637506 13752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:45:24.637578 13752 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:45:24.637602 13752 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39021/jwks.json
I20260809 04:45:24.643254 13752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:46227
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39021/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:24.644691 13752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:24.645933 13752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:45:24.655447 13752 server_base.cc:1034] running on GCE node
W20260809 04:45:24.655369 13759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:24.655412 13760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:24.655369 13762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:24.656260 13752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:24.657330 13752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:24.658511 13752 hybrid_clock.cc:648] HybridClock initialized: now 1786250724658462 us; error 66 us; skew 500 ppm
I20260809 04:45:24.660745 13752 webserver.cc:460] Webserver started at http://127.7.171.62:41181/ using document root <none> and password file <none>
I20260809 04:45:24.661392 13752 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:24.661465 13752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:24.661677 13752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:24.663858 13752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "52cd1d6ed3bf45ffb5816cb157c4e7f2"
format_stamp: "Formatted at 2026-08-09 04:45:24 on dist-test-slave-1zqn"
server_key: "d47c963038f8f4502caee51f4789fa26"
server_key_iv: "87fa0f46887b110a966c2b4ce7858253"
server_key_version: "encryptionkey@0"
I20260809 04:45:24.664526 13752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "52cd1d6ed3bf45ffb5816cb157c4e7f2"
format_stamp: "Formatted at 2026-08-09 04:45:24 on dist-test-slave-1zqn"
server_key: "d47c963038f8f4502caee51f4789fa26"
server_key_iv: "87fa0f46887b110a966c2b4ce7858253"
server_key_version: "encryptionkey@0"
I20260809 04:45:24.668396 13752 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:24.671074 13768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:24.672541 13752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:24.672684 13752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "52cd1d6ed3bf45ffb5816cb157c4e7f2"
format_stamp: "Formatted at 2026-08-09 04:45:24 on dist-test-slave-1zqn"
server_key: "d47c963038f8f4502caee51f4789fa26"
server_key_iv: "87fa0f46887b110a966c2b4ce7858253"
server_key_version: "encryptionkey@0"
I20260809 04:45:24.672796 13752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:24.694072 13752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:24.694896 13752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:24.695137 13752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:24.703636 13752 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:46227
I20260809 04:45:24.703653 13834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:46227 every 8 connection(s)
I20260809 04:45:24.704880 13752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:24.708521 13835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:24.711613 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 13752
I20260809 04:45:24.711757 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:24.712160 7852 external_mini_cluster.cc:1428] Setting key fe56bc1a12d2de7a0684cf356da3d00c
I20260809 04:45:24.715329 13835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2: Bootstrap starting.
I20260809 04:45:24.718341 13835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:24.719345 13835 log.cc:824] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:24.721807 13835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2: No bootstrap required, opened a new log
I20260809 04:45:24.724387 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:24.713700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58174 (local address 127.7.171.62:46227)
0809 04:45:24.714547 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:24.714572 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:24.714620 (+ 48us) server_negotiation.cc:407] Connection header received
0809 04:45:24.715570 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:24.715600 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:24.716169 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:24.716582 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:24.717092 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.718813 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.720325 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.721003 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.721178 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:24.722206 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:24.722213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:24.722230 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:24.722731 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:24.722800 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:24.722827 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:24.722930 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:24.723146 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 04:45:24.723425 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":201,"threads_started":1}
I20260809 04:45:24.726287 13835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cd1d6ed3bf45ffb5816cb157c4e7f2" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46227 } }
I20260809 04:45:24.726562 13835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:24.726640 13835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52cd1d6ed3bf45ffb5816cb157c4e7f2, State: Initialized, Role: FOLLOWER
I20260809 04:45:24.727106 13835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cd1d6ed3bf45ffb5816cb157c4e7f2" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46227 } }
I20260809 04:45:24.727264 13835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:24.727339 13835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:24.727442 13835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:24.727476 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:45:24.728488 13835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cd1d6ed3bf45ffb5816cb157c4e7f2" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46227 } }
I20260809 04:45:24.728864 13835 leader_election.cc:302] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52cd1d6ed3bf45ffb5816cb157c4e7f2; no voters:
I20260809 04:45:24.729214 13835 leader_election.cc:288] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:24.729355 13842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:24.729652 13842 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [term 1 LEADER]: Becoming Leader. State: Replica: 52cd1d6ed3bf45ffb5816cb157c4e7f2, State: Running, Role: LEADER
I20260809 04:45:24.730000 13842 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cd1d6ed3bf45ffb5816cb157c4e7f2" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46227 } }
I20260809 04:45:24.730440 13835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:24.731693 13843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52cd1d6ed3bf45ffb5816cb157c4e7f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cd1d6ed3bf45ffb5816cb157c4e7f2" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46227 } } }
I20260809 04:45:24.731827 13843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:24.732229 13844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52cd1d6ed3bf45ffb5816cb157c4e7f2. Latest consensus state: current_term: 1 leader_uuid: "52cd1d6ed3bf45ffb5816cb157c4e7f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cd1d6ed3bf45ffb5816cb157c4e7f2" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46227 } } }
I20260809 04:45:24.732352 13844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:24.732503 13853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:24.735744 13853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:24.741904 13853 catalog_manager.cc:1347] Generated new cluster ID: aa87e64dd8774de5aaf6fa688b543d4b
I20260809 04:45:24.741990 13853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:24.760787 13853 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:24.761749 13853 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:24.769784 13853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52cd1d6ed3bf45ffb5816cb157c4e7f2: Generated new TSK 0
I20260809 04:45:24.770505 13853 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 04:45:24.803076 13872 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 04:45:24.803201 13872 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 04:45:24.803972 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:24.787350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58188 (local address 127.7.171.62:46227)
0809 04:45:24.787513 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:24.787519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:24.787626 (+ 107us) server_negotiation.cc:407] Connection header received
0809 04:45:24.787781 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:24.787785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:24.787850 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:24.788147 (+ 297us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:45:24.788350 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.789185 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.790043 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.790382 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.790504 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:24.790845 (+ 341us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:45:24.803477 (+ 12632us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39021/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 04:45:24.803783 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58188: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39021/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 04:45:24.804013 13871 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:24.787153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:46227 (local address 127.0.0.1:58188)
0809 04:45:24.787536 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:24.787548 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:24.787639 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:24.788135 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:24.788137 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:24.788143 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:45:24.788242 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:24.788247 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.789293 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.789296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:24.789927 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:24.789935 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:24.790535 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:24.790540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:24.790627 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:24.790652 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:45:24.803698 (+ 13046us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39021/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 04:45:24.803847 (+ 149us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.171.62:46227: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39021/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":113,"threads_started":1}
W20260809 04:45:24.804103 13840 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58188: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39021/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260809 04:45:24.804776 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 13752
2026-08-09T04:45:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (345 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 04:45:24.848429 7852 mini_oidc.cc:152] Starting JWKS server
I20260809 04:45:24.849381 7852 webserver.cc:460] Webserver started at https://127.0.0.1:40671/ using document root <none> and password file <none>
I20260809 04:45:24.849509 7852 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:45:24.849809 7852 webserver.cc:460] Webserver started at http://127.0.0.1:37961/ using document root <none> and password file <none>
2026-08-09T04:45:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:24Z Disabled control of system clock
I20260809 04:45:24.860514 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:33271
--webserver_interface=127.7.171.62
--webserver_port=0
--builtin_ntp_servers=127.7.171.20:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:33271
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40671/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 04:45:24.974088 13879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:24.974355 13879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:24.974395 13879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:24.978390 13879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:24.978447 13879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:24.978462 13879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:24.978479 13879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:24.978494 13879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:45:24.978510 13879 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:45:24.978523 13879 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40671/jwks.json
I20260809 04:45:24.983203 13879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:33271
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:33271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40671/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:24.984411 13879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:24.985863 13879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:24.995087 13886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:24.995193 13879 server_base.cc:1034] running on GCE node
W20260809 04:45:24.995128 13889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:24.995087 13887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:24.995972 13879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:24.997231 13879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:24.998451 13879 hybrid_clock.cc:648] HybridClock initialized: now 1786250724998422 us; error 47 us; skew 500 ppm
I20260809 04:45:25.000692 13879 webserver.cc:460] Webserver started at http://127.7.171.62:44835/ using document root <none> and password file <none>
I20260809 04:45:25.001291 13879 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:25.001389 13879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:25.001641 13879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:25.003773 13879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "82f45d8d3c604961b57822da4de2010f"
format_stamp: "Formatted at 2026-08-09 04:45:24 on dist-test-slave-1zqn"
server_key: "64f70bf48a09328a73dd8bb15aafb8bb"
server_key_iv: "e69a3a68017756dc18b203a1b84a24db"
server_key_version: "encryptionkey@0"
I20260809 04:45:25.004729 13879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "82f45d8d3c604961b57822da4de2010f"
format_stamp: "Formatted at 2026-08-09 04:45:24 on dist-test-slave-1zqn"
server_key: "64f70bf48a09328a73dd8bb15aafb8bb"
server_key_iv: "e69a3a68017756dc18b203a1b84a24db"
server_key_version: "encryptionkey@0"
I20260809 04:45:25.009078 13879 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:25.011822 13896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:25.013235 13879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:25.013396 13879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "82f45d8d3c604961b57822da4de2010f"
format_stamp: "Formatted at 2026-08-09 04:45:24 on dist-test-slave-1zqn"
server_key: "64f70bf48a09328a73dd8bb15aafb8bb"
server_key_iv: "e69a3a68017756dc18b203a1b84a24db"
server_key_version: "encryptionkey@0"
I20260809 04:45:25.013538 13879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:25.036892 13879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:25.037712 13879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:25.037940 13879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:25.046087 13879 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:33271
I20260809 04:45:25.046175 13960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:33271 every 8 connection(s)
I20260809 04:45:25.047197 13879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:25.050724 13961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:25.056730 13961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f: Bootstrap starting.
I20260809 04:45:25.057394 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 13879
I20260809 04:45:25.057530 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:25.057854 7852 external_mini_cluster.cc:1428] Setting key 4edd21dea02318a059f7a19b70859291
I20260809 04:45:25.059932 13961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:25.060779 13961 log.cc:824] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:25.063125 13961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f: No bootstrap required, opened a new log
I20260809 04:45:25.067052 13961 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f45d8d3c604961b57822da4de2010f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33271 } }
I20260809 04:45:25.067395 13961 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:25.067495 13961 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82f45d8d3c604961b57822da4de2010f, State: Initialized, Role: FOLLOWER
I20260809 04:45:25.068046 13961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f45d8d3c604961b57822da4de2010f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33271 } }
I20260809 04:45:25.068250 13961 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:25.068284 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:25.059325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.7.171.62:33271)
0809 04:45:25.060184 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:25.060202 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:25.060245 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:45:25.060972 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:25.060992 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:25.061525 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:25.061956 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:25.062415 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.063494 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.064590 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.065087 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.065269 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:25.066190 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:25.066195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:25.066208 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:25.066657 (+ 449us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:25.066717 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:25.066737 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:25.066852 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:25.066998 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 04:45:25.067276 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":150,"threads_started":1}
I20260809 04:45:25.068342 13961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:25.068475 13961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:25.069763 13961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f45d8d3c604961b57822da4de2010f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33271 } }
I20260809 04:45:25.070147 13961 leader_election.cc:302] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82f45d8d3c604961b57822da4de2010f; no voters:
I20260809 04:45:25.070549 13961 leader_election.cc:288] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:25.070698 13966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:25.070986 13966 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [term 1 LEADER]: Becoming Leader. State: Replica: 82f45d8d3c604961b57822da4de2010f, State: Running, Role: LEADER
I20260809 04:45:25.071303 13966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f45d8d3c604961b57822da4de2010f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33271 } }
I20260809 04:45:25.071914 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:45:25.072093 13961 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:25.073550 13968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82f45d8d3c604961b57822da4de2010f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f45d8d3c604961b57822da4de2010f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33271 } } }
I20260809 04:45:25.073699 13968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:25.074546 13967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82f45d8d3c604961b57822da4de2010f. Latest consensus state: current_term: 1 leader_uuid: "82f45d8d3c604961b57822da4de2010f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82f45d8d3c604961b57822da4de2010f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33271 } } }
I20260809 04:45:25.074666 13967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f [sys.catalog]: This master's current role is: LEADER
W20260809 04:45:25.077309 13983 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:45:25.077399 13983 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:45:25.077513 13975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:25.078349 13975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:25.084622 13975 catalog_manager.cc:1347] Generated new cluster ID: 44844b6b4ffb4a768754f67fd4a0c5d5
I20260809 04:45:25.084705 13975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:25.101475 13975 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:25.102917 13975 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:25.109309 13975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82f45d8d3c604961b57822da4de2010f: Generated new TSK 0
I20260809 04:45:25.110227 13975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:45:25.134227 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:25.128426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55838 (local address 127.7.171.62:33271)
0809 04:45:25.128585 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:25.128590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:25.128800 (+ 210us) server_negotiation.cc:407] Connection header received
0809 04:45:25.128946 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:25.128950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:25.129009 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:25.129130 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:45:25.129384 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.130245 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.131360 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.131719 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.131824 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:25.132280 (+ 456us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:45:25.133815 (+ 1535us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 04:45:25.134063 (+ 248us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55838: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 04:45:25.134310 13964 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55838: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 04:45:25.136281 13993 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:25.128346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:33271 (local address 127.0.0.1:55838)
0809 04:45:25.128703 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:25.128719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:25.128824 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:25.129141 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:25.129144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:25.129150 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:45:25.129275 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:25.129280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.130419 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.130423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:25.131229 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:25.131239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.131853 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.131857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:25.131933 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:25.132009 (+ 76us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:45:25.134009 (+ 2000us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 04:45:25.136083 (+ 2074us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.171.62:33271: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":1831,"thread_start_us":162,"threads_started":1}
I20260809 04:45:25.137092 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 13879
E20260809 04:45:25.149405 13994 webserver.cc:533] Webserver: sslize failed:
E20260809 04:45:25.149506 13994 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T04:45:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (332 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T04:45:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:25Z Disabled control of system clock
I20260809 04:45:25.191283 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:33867
--webserver_interface=127.7.171.62
--webserver_port=0
--builtin_ntp_servers=127.7.171.20:33843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:33867
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/testchainca.cert with env {}
W20260809 04:45:25.309458 13999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:25.309723 13999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:25.309764 13999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:25.314005 13999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:25.314069 13999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:25.314085 13999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:25.314101 13999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:25.314116 13999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:45:25.314132 13999 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:45:25.314146 13999 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 04:45:25.319556 13999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:33843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:33867
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:33867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:25.320623 13999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:25.321620 13999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:25.330649 14005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:25.330710 14004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:25.330685 14007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:25.330811 13999 server_base.cc:1034] running on GCE node
I20260809 04:45:25.331436 13999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:25.332551 13999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:25.333720 13999 hybrid_clock.cc:648] HybridClock initialized: now 1786250725333706 us; error 25 us; skew 500 ppm
I20260809 04:45:25.335631 13999 webserver.cc:460] Webserver started at http://127.7.171.62:40413/ using document root <none> and password file <none>
I20260809 04:45:25.336226 13999 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:25.336294 13999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:25.336457 13999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:25.338394 13999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "4fe97d8f601f492ca69c343a6d325d8b"
format_stamp: "Formatted at 2026-08-09 04:45:25 on dist-test-slave-1zqn"
server_key: "836b41bc27814860a3a168b51d420d95"
server_key_iv: "2e04dd0b31c7c097a8347eee68bad8c8"
server_key_version: "encryptionkey@0"
I20260809 04:45:25.338917 13999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "4fe97d8f601f492ca69c343a6d325d8b"
format_stamp: "Formatted at 2026-08-09 04:45:25 on dist-test-slave-1zqn"
server_key: "836b41bc27814860a3a168b51d420d95"
server_key_iv: "2e04dd0b31c7c097a8347eee68bad8c8"
server_key_version: "encryptionkey@0"
I20260809 04:45:25.342679 13999 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 04:45:25.345259 14015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:25.346517 13999 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:25.346659 13999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "4fe97d8f601f492ca69c343a6d325d8b"
format_stamp: "Formatted at 2026-08-09 04:45:25 on dist-test-slave-1zqn"
server_key: "836b41bc27814860a3a168b51d420d95"
server_key_iv: "2e04dd0b31c7c097a8347eee68bad8c8"
server_key_version: "encryptionkey@0"
I20260809 04:45:25.346776 13999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:25.368333 13999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:25.369081 13999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:25.369289 13999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:25.376634 13999 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:33867
I20260809 04:45:25.376654 14074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:33867 every 8 connection(s)
I20260809 04:45:25.377574 13999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:25.378304 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 13999
I20260809 04:45:25.378397 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:25.378710 7852 external_mini_cluster.cc:1428] Setting key a9416b960dab624a898b429f376827bf
I20260809 04:45:25.381202 14075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:25.388749 14075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b: Bootstrap starting.
I20260809 04:45:25.388813 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:25.380126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.7.171.62:33867)
0809 04:45:25.380775 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:25.380791 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:25.380828 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:45:25.381543 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:25.381559 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:25.381944 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:25.382227 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:25.382796 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.383803 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.385044 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.385519 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.385642 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:25.386606 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:25.386611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:25.386623 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:25.387080 (+ 457us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:25.387139 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:25.387160 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:25.387244 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:25.387441 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 04:45:25.387761 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":149,"threads_started":1}
I20260809 04:45:25.391215 14075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:25.392189 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:45:25.392174 14075 log.cc:824] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:25.394469 14075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b: No bootstrap required, opened a new log
I20260809 04:45:25.398417 14075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe97d8f601f492ca69c343a6d325d8b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33867 } }
I20260809 04:45:25.398697 14075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:25.398787 14075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fe97d8f601f492ca69c343a6d325d8b, State: Initialized, Role: FOLLOWER
I20260809 04:45:25.399282 14075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe97d8f601f492ca69c343a6d325d8b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33867 } }
I20260809 04:45:25.399510 14075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:25.399619 14075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:25.399757 14075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:25.400861 14075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe97d8f601f492ca69c343a6d325d8b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33867 } }
I20260809 04:45:25.401232 14075 leader_election.cc:302] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fe97d8f601f492ca69c343a6d325d8b; no voters:
I20260809 04:45:25.401587 14075 leader_election.cc:288] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:25.401721 14083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:25.401983 14083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [term 1 LEADER]: Becoming Leader. State: Replica: 4fe97d8f601f492ca69c343a6d325d8b, State: Running, Role: LEADER
I20260809 04:45:25.402289 14083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe97d8f601f492ca69c343a6d325d8b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33867 } }
I20260809 04:45:25.402895 14075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:25.404088 14085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fe97d8f601f492ca69c343a6d325d8b. Latest consensus state: current_term: 1 leader_uuid: "4fe97d8f601f492ca69c343a6d325d8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe97d8f601f492ca69c343a6d325d8b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33867 } } }
I20260809 04:45:25.404250 14085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:25.404605 14084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fe97d8f601f492ca69c343a6d325d8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fe97d8f601f492ca69c343a6d325d8b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 33867 } } }
I20260809 04:45:25.404757 14084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:25.404659 14092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:25.408196 14092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:25.415036 14092 catalog_manager.cc:1347] Generated new cluster ID: 2ba15a90e1d6499eb4e2993bc16850aa
I20260809 04:45:25.415143 14092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:25.432693 14092 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:25.433660 14092 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:25.443830 14092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fe97d8f601f492ca69c343a6d325d8b: Generated new TSK 0
I20260809 04:45:25.444645 14092 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:45:25.524609 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:25.461916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46158 (local address 127.7.171.62:33867)
0809 04:45:25.462065 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:25.462071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:25.462253 (+ 182us) server_negotiation.cc:407] Connection header received
0809 04:45:25.462436 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:25.462438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:25.462484 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:25.462568 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:45:25.462836 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.463360 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.464616 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.464837 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.464920 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:25.465239 (+ 319us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:45:25.524168 (+ 58929us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 04:45:25.524477 (+ 309us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46158: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 04:45:25.524721 14079 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46158: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 04:45:25.524876 14110 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:25.461843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:33867 (local address 127.0.0.1:46158)
0809 04:45:25.462159 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:25.462175 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:25.462294 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:25.462595 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:25.462598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:25.462606 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:45:25.462730 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:25.462737 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.463474 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.463478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:25.464497 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:25.464508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:25.464960 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:25.464964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:25.465051 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:25.465079 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:45:25.524446 (+ 59367us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 04:45:25.524679 (+ 233us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.171.62:33867: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":116,"threads_started":1}
I20260809 04:45:25.525702 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 13999
2026-08-09T04:45:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (368 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (128 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (128 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:25 dist-test-slave-1zqn krb5kdc[14124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:25 dist-test-slave-1zqn krb5kdc[14124](info): set up 2 sockets
Aug 09 04:45:25 dist-test-slave-1zqn krb5kdc[14124](info): commencing operation
krb5kdc: starting...
W20260809 04:45:26.903412 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:27 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:27.053489 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34221
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:34221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:27.168346 14137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:27.168632 14137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:27.168674 14137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:27.172863 14137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:27.172921 14137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:27.172940 14137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:27.172957 14137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:27.173019 14137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:27.178339 14137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:34221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:27.179440 14137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:27.180517 14137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:27.189235 14143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:27.189281 14142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:27.189280 14147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:27.189332 14137 server_base.cc:1034] running on GCE node
I20260809 04:45:27.190055 14137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:27.191272 14137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:27.192454 14137 hybrid_clock.cc:648] HybridClock initialized: now 1786250727192414 us; error 50 us; skew 500 ppm
Aug 09 04:45:27 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:27.201716 14137 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:45:27.202806 14137 webserver.cc:460] Webserver started at http://127.7.171.62:36109/ using document root <none> and password file <none>
I20260809 04:45:27.203373 14137 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:27.203459 14137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:27.203644 14137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:27.205919 14137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "d43196a059f646ec971a4375bda9ce38"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "00bd037250bd42b3ddd61cf991041200"
server_key_iv: "0e545cc46b00fc8072acf767091c444b"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.206555 14137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "d43196a059f646ec971a4375bda9ce38"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "00bd037250bd42b3ddd61cf991041200"
server_key_iv: "0e545cc46b00fc8072acf767091c444b"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.210541 14137 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260809 04:45:27.213090 14156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:27.214537 14137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:27.214706 14137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "d43196a059f646ec971a4375bda9ce38"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "00bd037250bd42b3ddd61cf991041200"
server_key_iv: "0e545cc46b00fc8072acf767091c444b"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.214830 14137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:27.227371 14137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:27.230852 14137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:27.231104 14137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:27.238732 14137 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:34221
I20260809 04:45:27.238772 14218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:34221 every 8 connection(s)
I20260809 04:45:27.239684 14137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:27.240751 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 14137
I20260809 04:45:27.240897 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:27.241302 7852 external_mini_cluster.cc:1428] Setting key 2a9729587a976899f7fc36d3bb2e382a
I20260809 04:45:27.242898 14219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:27.249627 14219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38: Bootstrap starting.
Aug 09 04:45:27 dist-test-slave-1zqn krb5kdc[14124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:27.252178 14219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:27.253054 14219 log.cc:824] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:27.254954 14219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38: No bootstrap required, opened a new log
I20260809 04:45:27.256669 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:27.242775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42504 (local address 127.7.171.62:34221)
0809 04:45:27.243285 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:27.243310 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:27.243350 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:45:27.244007 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:27.244022 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:27.244408 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:27.244687 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:27.245634 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:27.246689 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:27.247832 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:27.248394 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:27.248523 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:27.251513 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:27.251518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:27.251532 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:27.251564 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:27.253551 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:27.254038 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:27.254043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:27.254047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:27.254164 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:27.254459 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:27.254464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:27.254466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:27.254863 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:27.255061 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:27.255371 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 04:45:27.255668 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":253,"threads_started":1}
I20260809 04:45:27.258164 14219 raft_consensus.cc:348] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43196a059f646ec971a4375bda9ce38" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34221 } }
I20260809 04:45:27.258404 14219 raft_consensus.cc:374] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:27.258486 14219 raft_consensus.cc:729] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d43196a059f646ec971a4375bda9ce38, State: Initialized, Role: FOLLOWER
I20260809 04:45:27.258985 14219 consensus_queue.cc:260] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43196a059f646ec971a4375bda9ce38" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34221 } }
I20260809 04:45:27.259145 14219 raft_consensus.cc:388] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:27.259235 14219 raft_consensus.cc:482] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:27.259354 14219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:27.260366 14219 raft_consensus.cc:504] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43196a059f646ec971a4375bda9ce38" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34221 } }
I20260809 04:45:27.260707 14219 leader_election.cc:302] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d43196a059f646ec971a4375bda9ce38; no voters:
I20260809 04:45:27.261008 14219 leader_election.cc:288] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:27.261180 14225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:27.261469 14225 raft_consensus.cc:686] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [term 1 LEADER]: Becoming Leader. State: Replica: d43196a059f646ec971a4375bda9ce38, State: Running, Role: LEADER
I20260809 04:45:27.261817 14225 consensus_queue.cc:237] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43196a059f646ec971a4375bda9ce38" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34221 } }
I20260809 04:45:27.262295 14219 sys_catalog.cc:564] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:27.263726 14227 sys_catalog.cc:455] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d43196a059f646ec971a4375bda9ce38. Latest consensus state: current_term: 1 leader_uuid: "d43196a059f646ec971a4375bda9ce38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43196a059f646ec971a4375bda9ce38" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34221 } } }
I20260809 04:45:27.263873 14227 sys_catalog.cc:458] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:27.264206 14226 sys_catalog.cc:455] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d43196a059f646ec971a4375bda9ce38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43196a059f646ec971a4375bda9ce38" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34221 } } }
I20260809 04:45:27.264323 14226 sys_catalog.cc:458] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:27.265133 14234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:27.268189 14234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:27.273887 14234 catalog_manager.cc:1347] Generated new cluster ID: 82cff8ffbd1e4412bf0b41c7a83c88ec
I20260809 04:45:27.274051 14234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:27.300109 14234 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:27.301137 14234 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:27.307585 14234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d43196a059f646ec971a4375bda9ce38: Generated new TSK 0
I20260809 04:45:27.308420 14234 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:45:27.339288 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34221
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:27.481982 14247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:27.482282 14247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:27.482327 14247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:27.486706 14247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:27.486773 14247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:27.486835 14247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:27.492093 14247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:27.493206 14247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:27.494239 14247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:27.505054 14253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:27.505064 14256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:27.505064 14254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:27.505511 14247 server_base.cc:1034] running on GCE node
I20260809 04:45:27.505889 14247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:27.506500 14247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:27.507664 14247 hybrid_clock.cc:648] HybridClock initialized: now 1786250727507632 us; error 43 us; skew 500 ppm
Aug 09 04:45:27 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:27.510485 14247 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:45:27.511416 14247 webserver.cc:460] Webserver started at http://127.7.171.1:37205/ using document root <none> and password file <none>
I20260809 04:45:27.512007 14247 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:27.512137 14247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:27.512389 14247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:27.514483 14247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "07d707f45ff94dbc9c507dcaee7624af"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "53f8a6e6093cb90573ef9aeecf740850"
server_key_iv: "2863038d93c682f65b058725a9848266"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.515079 14247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "07d707f45ff94dbc9c507dcaee7624af"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "53f8a6e6093cb90573ef9aeecf740850"
server_key_iv: "2863038d93c682f65b058725a9848266"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.519294 14247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260809 04:45:27.521914 14265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:27.523273 14247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:45:27.523440 14247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "07d707f45ff94dbc9c507dcaee7624af"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "53f8a6e6093cb90573ef9aeecf740850"
server_key_iv: "2863038d93c682f65b058725a9848266"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.523566 14247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:27.542177 14247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:27.545784 14247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:27.546013 14247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:27.546640 14247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:27.547555 14247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:27.547626 14247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:27.547688 14247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:27.547739 14247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:27.558516 14247 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:45693
I20260809 04:45:27.558588 14398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:45693 every 8 connection(s)
I20260809 04:45:27.559720 14247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:27.568209 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 14247
I20260809 04:45:27.568353 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:27.568668 7852 external_mini_cluster.cc:1428] Setting key 79d28ccc2316932f59c5b0c4e55e227a
Aug 09 04:45:27 dist-test-slave-1zqn krb5kdc[14124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:27.575774 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:27.561861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:41041 (local address 127.7.171.62:34221)
0809 04:45:27.561993 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:27.561997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:27.563071 (+ 1074us) server_negotiation.cc:407] Connection header received
0809 04:45:27.564696 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:27.564699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:27.564757 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:27.564872 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:27.566136 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:27.566703 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:27.567921 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:27.568240 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:27.568356 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:27.571495 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:27.571502 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:27.571508 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:27.571544 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:27.573113 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:27.573776 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:27.573780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:27.573781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:27.573861 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:27.574317 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:27.574324 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:27.574329 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:27.574642 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:27.574774 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:27.575292 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 04:45:27.575479 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 04:45:27.576429 14401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:27.562320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34221 (local address 127.7.171.1:41041)
0809 04:45:27.562882 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:27.562942 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:27.564474 (+ 1532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:27.565066 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:27.565079 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:27.565476 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:27.565971 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:27.565984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:27.566843 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:27.566846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:27.567777 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:27.567786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:27.568409 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:27.568413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:27.568593 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:27.569342 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:27.569363 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:27.571303 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:27.573241 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:27.573249 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:27.573270 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:27.573662 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:27.573959 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:27.573964 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:27.573969 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:27.574169 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:27.574762 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:27.574767 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:27.575012 (+ 245us) client_negotiation.cc:769] Sending connection context
0809 04:45:27.575199 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 04:45:27.575471 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":167,"threads_started":1}
I20260809 04:45:27.577745 14399 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34221
I20260809 04:45:27.578016 14399 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:27.578532 14399 heartbeater.cc:511] Master 127.7.171.62:34221 requested a full tablet report, sending...
I20260809 04:45:27.580207 14177 ts_manager.cc:194] Registered new tserver with Master: 07d707f45ff94dbc9c507dcaee7624af (127.7.171.1:45693)
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:45:27.581322 14177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:41041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:45:27.604357 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34221
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:27.719997 14404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:27.720330 14404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:27.720414 14404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:27.724570 14404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:27.724648 14404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:27.724735 14404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:27.730118 14404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:27.731158 14404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:27.732272 14404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:27.742815 14414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:27.742918 14416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:27.743059 14412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:27.743235 14404 server_base.cc:1034] running on GCE node
I20260809 04:45:27.743695 14404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:27.744398 14404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:27.745591 14404 hybrid_clock.cc:648] HybridClock initialized: now 1786250727745577 us; error 27 us; skew 500 ppm
Aug 09 04:45:27 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:27.748844 14404 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:45:27.749773 14404 webserver.cc:460] Webserver started at http://127.7.171.2:42675/ using document root <none> and password file <none>
I20260809 04:45:27.750320 14404 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:27.750379 14404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:27.750550 14404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:27.752839 14404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "c2ebb42d0ea549768d8505adc01f0387"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "3e14dfc5bd0b2c12cffb2714f4015db0"
server_key_iv: "34afb804702dabcd1212028104a23107"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.753382 14404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "c2ebb42d0ea549768d8505adc01f0387"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "3e14dfc5bd0b2c12cffb2714f4015db0"
server_key_iv: "34afb804702dabcd1212028104a23107"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.757596 14404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:27.760269 14427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:27.761647 14404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:45:27.761788 14404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "c2ebb42d0ea549768d8505adc01f0387"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "3e14dfc5bd0b2c12cffb2714f4015db0"
server_key_iv: "34afb804702dabcd1212028104a23107"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.761941 14404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:27.775729 14404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:27.778744 14404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:27.778908 14404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:27.779461 14404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:27.780368 14404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:27.780421 14404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:27.780464 14404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:27.780485 14404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:27.790645 14404 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:33303
I20260809 04:45:27.790668 14561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:33303 every 8 connection(s)
I20260809 04:45:27.791575 14404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:27.792201 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 14404
I20260809 04:45:27.792327 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:27.792598 7852 external_mini_cluster.cc:1428] Setting key 143ef5ef97210638e5d10d3ede2b779a
Aug 09 04:45:27 dist-test-slave-1zqn krb5kdc[14124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
I20260809 04:45:27.807410 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:27.793980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:46017 (local address 127.7.171.62:34221)
0809 04:45:27.794156 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:27.794161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:27.795231 (+ 1070us) server_negotiation.cc:407] Connection header received
0809 04:45:27.796746 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:27.796749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:27.796803 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:27.796891 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:27.798133 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:27.798685 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:27.799972 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:27.800295 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:27.800416 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:27.803032 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:27.803036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:27.803040 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:27.803082 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:27.804889 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:27.805538 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:27.805542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:27.805543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:27.805623 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:27.806099 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:27.806102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:27.806103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:27.806375 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:27.806506 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:27.807067 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 04:45:27.807220 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:45:27.808535 14566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:27.794511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34221 (local address 127.7.171.2:46017)
0809 04:45:27.795076 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:27.795125 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:27.796516 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:27.797094 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:27.797101 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:27.797471 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:27.797968 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:27.797981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:27.798840 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:27.798844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:27.799839 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:27.799849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:27.800462 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:27.800466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:27.800649 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:27.801310 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:27.801326 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:27.802838 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:27.805028 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:27.805035 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:27.805053 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:27.805426 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:27.805720 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:27.805723 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:27.805726 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:27.805938 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:27.806499 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:27.806510 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:27.806807 (+ 297us) client_negotiation.cc:769] Sending connection context
0809 04:45:27.807017 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 04:45:27.807375 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":161,"threads_started":1}
I20260809 04:45:27.809949 14564 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34221
I20260809 04:45:27.810238 14564 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:27.810840 14564 heartbeater.cc:511] Master 127.7.171.62:34221 requested a full tablet report, sending...
I20260809 04:45:27.812001 14177 ts_manager.cc:194] Registered new tserver with Master: c2ebb42d0ea549768d8505adc01f0387 (127.7.171.2:33303)
I20260809 04:45:27.812681 14177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:46017
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:45:27.825558 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34221
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:27.942807 14569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:27.943063 14569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:27.943109 14569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:27.947192 14569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:27.947254 14569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:27.947357 14569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:27.952854 14569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:27.953907 14569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:27.954958 14569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:27.964772 14575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:27.964768 14580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:27.964921 14569 server_base.cc:1034] running on GCE node
W20260809 04:45:27.964742 14576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:27.965461 14569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:27.966169 14569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:27.967351 14569 hybrid_clock.cc:648] HybridClock initialized: now 1786250727967275 us; error 86 us; skew 500 ppm
Aug 09 04:45:27 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:27.970321 14569 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:45:27.971261 14569 webserver.cc:460] Webserver started at http://127.7.171.3:45493/ using document root <none> and password file <none>
I20260809 04:45:27.971825 14569 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:27.971889 14569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:27.972203 14569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:27.974301 14569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "315bb969566b4cf68003fd022a5d1aee"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "ba403a456d6880940553502113165deb"
server_key_iv: "7ff5f18f514038a83f87af802aa76f92"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.974874 14569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "315bb969566b4cf68003fd022a5d1aee"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "ba403a456d6880940553502113165deb"
server_key_iv: "7ff5f18f514038a83f87af802aa76f92"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.978652 14569 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260809 04:45:27.981196 14587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:27.982421 14569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:27.982597 14569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "315bb969566b4cf68003fd022a5d1aee"
format_stamp: "Formatted at 2026-08-09 04:45:27 on dist-test-slave-1zqn"
server_key: "ba403a456d6880940553502113165deb"
server_key_iv: "7ff5f18f514038a83f87af802aa76f92"
server_key_version: "encryptionkey@0"
I20260809 04:45:27.982717 14569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:28.003499 14569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:28.007633 14569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:28.007879 14569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:28.008669 14569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:28.009609 14569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:28.009687 14569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:28.009755 14569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:28.009815 14569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:28.020632 14569 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:38097
I20260809 04:45:28.020664 14721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:38097 every 8 connection(s)
I20260809 04:45:28.021567 14569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:28.023690 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 14569
I20260809 04:45:28.023823 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:28.024206 7852 external_mini_cluster.cc:1428] Setting key 906a106f4742aabe2f797a0b393c77c1
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:28.035909 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:28.023529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:42443 (local address 127.7.171.62:34221)
0809 04:45:28.023720 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:28.023726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:28.024749 (+ 1023us) server_negotiation.cc:407] Connection header received
0809 04:45:28.026400 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:28.026406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:28.026459 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:28.026550 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:28.027856 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:28.028476 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:28.029632 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:28.029839 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:28.029916 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:28.032600 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:28.032604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:28.032606 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:28.032630 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:28.033980 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:28.034515 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:28.034517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:28.034518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:28.034593 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:28.034935 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:28.034937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:28.034938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:28.035127 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:28.035203 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:28.035684 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 04:45:28.035788 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:45:28.036818 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:28.023998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34221 (local address 127.7.171.3:42443)
0809 04:45:28.024587 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:28.024644 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:28.026159 (+ 1515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:28.026734 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:28.026741 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:28.027121 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:28.027706 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:28.027727 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:28.028599 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:28.028602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:28.029521 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:28.029531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:28.030002 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:28.030005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:28.030169 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:28.030870 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:28.030892 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:28.032437 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:28.034100 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:28.034105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:28.034116 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:28.034417 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:28.034701 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:28.034704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:28.034705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:28.034850 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:28.035221 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:28.035226 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:28.035469 (+ 243us) client_negotiation.cc:769] Sending connection context
0809 04:45:28.035628 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 04:45:28.035871 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":164,"threads_started":1}
I20260809 04:45:28.037884 14722 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34221
I20260809 04:45:28.038146 14722 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:28.038662 14722 heartbeater.cc:511] Master 127.7.171.62:34221 requested a full tablet report, sending...
I20260809 04:45:28.039690 14177 ts_manager.cc:194] Registered new tserver with Master: 315bb969566b4cf68003fd022a5d1aee (127.7.171.3:38097)
I20260809 04:45:28.040300 14177 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:42443
I20260809 04:45:28.050437 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/test-admin.keytab.
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:28.069180 7852 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 04:45:28.072149 14731 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:28.070664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36552 (local address 127.7.171.1:45693)
0809 04:45:28.070979 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:28.070995 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:28.071020 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:45:28.071072 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:28.071077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:28.071280 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:28.071405 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:28.071915 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36552: BlockingRecv error: recv got EOF from 127.0.0.1:36552 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":101,"threads_started":1}
W20260809 04:45:28.072196 7852 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5cc' of type FILE
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:28.073280 7852 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:28.074213 7852 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:45:28.084249 14731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:28.075103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36568 (local address 127.7.171.1:45693)
0809 04:45:28.075251 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:28.075261 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:28.075315 (+ 54us) server_negotiation.cc:407] Connection header received
0809 04:45:28.075416 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:28.075418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:28.075455 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:28.075522 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:28.076028 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:28.077131 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:28.077975 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:28.078389 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:28.078476 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:28.080799 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:28.080809 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:28.080823 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:28.080846 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:28.082383 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:28.082882 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:28.082888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:28.082889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:28.082966 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:28.083219 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:28.083221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:28.083222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:28.083461 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:28.083622 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:28.083846 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 04:45:28.083973 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 04:45:28.088486 14731 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:28.087348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36570 (local address 127.7.171.1:45693)
0809 04:45:28.087525 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:28.087528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:28.087637 (+ 109us) server_negotiation.cc:407] Connection header received
0809 04:45:28.087728 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:28.087730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:28.087768 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:28.087833 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:28.088350 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36570: BlockingRecv error: recv got EOF from 127.0.0.1:36570 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 04:45:28.088650 7852 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5cc' of type FILE
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:28.089726 7852 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:28.090656 7852 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:45:28.099685 14731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:28.091593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36576 (local address 127.7.171.1:45693)
0809 04:45:28.091710 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:28.091713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:28.091849 (+ 136us) server_negotiation.cc:407] Connection header received
0809 04:45:28.091940 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:28.091942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:28.091978 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:28.092046 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:28.092604 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:28.093114 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:28.093889 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:28.094194 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:28.094291 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:28.096616 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:28.096620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:28.096622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:28.096644 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:28.097896 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:28.098430 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:28.098434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:28.098435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:28.098522 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:28.098809 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:28.098812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:28.098813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:28.099054 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:28.099165 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:28.099357 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 04:45:28.099505 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 04:45:28.102451 14731 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:28.101529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.7.171.1:45693)
0809 04:45:28.101677 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:28.101681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:28.101728 (+ 47us) server_negotiation.cc:407] Connection header received
0809 04:45:28.101815 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:28.101817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:28.101875 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:28.101961 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:28.102341 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36580: BlockingRecv error: recv got EOF from 127.0.0.1:36580 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 04:45:28.102797 7852 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250697618759-7852-0/krb5kdc/krb5cc)
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:28.103859 7852 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:28.104851 7852 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:45:28.114039 14731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:28.105652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36592 (local address 127.7.171.1:45693)
0809 04:45:28.105776 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:28.105780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:28.105849 (+ 69us) server_negotiation.cc:407] Connection header received
0809 04:45:28.106072 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:28.106074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:28.106110 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:28.106199 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:28.106723 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:28.107253 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:28.108113 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:28.108337 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:28.108466 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:28.110702 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:28.110706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:28.110709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:28.110744 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:28.112179 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:28.112669 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:28.112675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:28.112677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:28.112786 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:28.113105 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:28.113109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:28.113111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:28.113303 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:28.113453 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:28.113720 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 04:45:28.113839 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:45:28.115129 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 14247
I20260809 04:45:28.126132 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 14404
I20260809 04:45:28.133050 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 14569
I20260809 04:45:28.139678 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 14137
2026-08-09T04:45:28Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2357 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14745](info): set up 2 sockets
Aug 09 04:45:28 dist-test-slave-1zqn krb5kdc[14745](info): commencing operation
krb5kdc: starting...
W20260809 04:45:29.267755 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:29 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:29Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.62.keytab.
Entry for principal oryx/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.62.keytab.
I20260809 04:45:29.403524 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:36799
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.62.keytab
--principal=oryx/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:36799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:29.520365 14759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:29.520686 14759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:29.520779 14759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:29.525018 14759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:29.525100 14759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:29.525138 14759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:29.525177 14759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:29.525236 14759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:29.530704 14759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:36799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.62.keytab
--principal=oryx/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:36799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:29.531872 14759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:29.532989 14759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:29.541719 14766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:29.541720 14769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:29.541742 14765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:29.542179 14759 server_base.cc:1034] running on GCE node
I20260809 04:45:29.542795 14759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:29.543915 14759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:29.545138 14759 hybrid_clock.cc:648] HybridClock initialized: now 1786250729545122 us; error 26 us; skew 500 ppm
Aug 09 04:45:29 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250729, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:29.548039 14759 init.cc:377] Logged in from keytab as oryx/127.7.171.62@KRBTEST.COM (short username oryx)
I20260809 04:45:29.548988 14759 webserver.cc:460] Webserver started at http://127.7.171.62:34123/ using document root <none> and password file <none>
I20260809 04:45:29.549539 14759 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:29.549603 14759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:29.549775 14759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:29.551797 14759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "a936256175c245da8101aab12ceb2c23"
format_stamp: "Formatted at 2026-08-09 04:45:29 on dist-test-slave-1zqn"
server_key: "d430675762eba6eb8f38043179eab026"
server_key_iv: "addd0e49da065330fe66c173a8559e7e"
server_key_version: "encryptionkey@0"
I20260809 04:45:29.552402 14759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "a936256175c245da8101aab12ceb2c23"
format_stamp: "Formatted at 2026-08-09 04:45:29 on dist-test-slave-1zqn"
server_key: "d430675762eba6eb8f38043179eab026"
server_key_iv: "addd0e49da065330fe66c173a8559e7e"
server_key_version: "encryptionkey@0"
I20260809 04:45:29.556273 14759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:45:29.558658 14777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:29.559923 14759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:29.560168 14759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "a936256175c245da8101aab12ceb2c23"
format_stamp: "Formatted at 2026-08-09 04:45:29 on dist-test-slave-1zqn"
server_key: "d430675762eba6eb8f38043179eab026"
server_key_iv: "addd0e49da065330fe66c173a8559e7e"
server_key_version: "encryptionkey@0"
I20260809 04:45:29.560290 14759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:29.595777 14759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:29.599404 14759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:29.599653 14759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:29.606974 14759 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:36799
I20260809 04:45:29.607276 14834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:36799 every 8 connection(s)
I20260809 04:45:29.608137 14759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:29.611090 14835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:29.611467 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 14759
I20260809 04:45:29.611624 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:29.611954 7852 external_mini_cluster.cc:1428] Setting key fe1a4d7d48c18cc1a5122e1b53c09a0c
I20260809 04:45:29.618427 14835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23: Bootstrap starting.
I20260809 04:45:29.621449 14835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23: Neither blocks nor log segments found. Creating new log.
Aug 09 04:45:29 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:29.622277 14835 log.cc:824] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:29.624971 14835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23: No bootstrap required, opened a new log
I20260809 04:45:29.628340 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:29.613478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38870 (local address 127.7.171.62:36799)
0809 04:45:29.613955 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:29.613971 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:29.614010 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:45:29.614698 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:29.614715 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:29.615065 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:29.615349 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:29.616357 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:29.617356 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:29.618550 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:29.619262 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:29.619409 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:29.622634 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:29.622643 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:29.622664 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:29.622728 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:29.624926 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:29.625432 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:29.625438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:29.625444 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:29.625597 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:29.625865 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:29.625868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:29.625870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:29.626389 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:29.626607 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:29.626888 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 04:45:29.627216 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":175,"threads_started":1}
I20260809 04:45:29.629832 14835 raft_consensus.cc:348] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } }
I20260809 04:45:29.630055 14835 raft_consensus.cc:374] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:29.630122 14835 raft_consensus.cc:729] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a936256175c245da8101aab12ceb2c23, State: Initialized, Role: FOLLOWER
I20260809 04:45:29.630648 14835 consensus_queue.cc:260] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } }
I20260809 04:45:29.630811 14835 raft_consensus.cc:388] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:29.630880 14835 raft_consensus.cc:482] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:29.630982 14835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:29.631958 14835 raft_consensus.cc:504] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } }
I20260809 04:45:29.632345 14835 leader_election.cc:302] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a936256175c245da8101aab12ceb2c23; no voters:
I20260809 04:45:29.632668 14835 leader_election.cc:288] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:29.632810 14840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:29.633082 14840 raft_consensus.cc:686] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 1 LEADER]: Becoming Leader. State: Replica: a936256175c245da8101aab12ceb2c23, State: Running, Role: LEADER
I20260809 04:45:29.633419 14840 consensus_queue.cc:237] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } }
I20260809 04:45:29.633946 14835 sys_catalog.cc:564] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:29.635231 14841 sys_catalog.cc:455] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } }
I20260809 04:45:29.635257 14843 sys_catalog.cc:455] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a936256175c245da8101aab12ceb2c23. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } }
I20260809 04:45:29.635365 14841 sys_catalog.cc:458] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:29.635366 14843 sys_catalog.cc:458] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:29.635774 14847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:29.638761 14847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:29.644223 14847 catalog_manager.cc:1347] Generated new cluster ID: 98ab283728104e609f7f6e43a857e25c
I20260809 04:45:29.644292 14847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:29.664296 14847 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:29.665422 14847 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:29.670594 14847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23: Generated new TSK 0
I20260809 04:45:29.671404 14847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab.
Entry for principal oryx/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab.
I20260809 04:45:29.712479 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36799
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab
--principal=oryx/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:29.835399 14863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:29.835698 14863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:29.835779 14863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:29.840134 14863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:29.840199 14863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:29.840304 14863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:29.845728 14863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab
--principal=oryx/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:29.846904 14863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:29.848003 14863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:29.858392 14870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:29.858486 14869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:29.858400 14872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:29.858819 14863 server_base.cc:1034] running on GCE node
I20260809 04:45:29.859272 14863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:29.859961 14863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:29.861207 14863 hybrid_clock.cc:648] HybridClock initialized: now 1786250729861188 us; error 30 us; skew 500 ppm
Aug 09 04:45:29 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250729, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:29.864749 14863 init.cc:377] Logged in from keytab as oryx/127.7.171.1@KRBTEST.COM (short username oryx)
I20260809 04:45:29.865684 14863 webserver.cc:460] Webserver started at http://127.7.171.1:34817/ using document root <none> and password file <none>
I20260809 04:45:29.866245 14863 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:29.866310 14863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:29.866496 14863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:29.868716 14863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "9e91f0d374ee4533b1ee5b6f0f016253"
format_stamp: "Formatted at 2026-08-09 04:45:29 on dist-test-slave-1zqn"
server_key: "2d4bccc5a92772d602d1b31fe67807fd"
server_key_iv: "09971f50f10ec21480e0b0899df3274b"
server_key_version: "encryptionkey@0"
I20260809 04:45:29.869284 14863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "9e91f0d374ee4533b1ee5b6f0f016253"
format_stamp: "Formatted at 2026-08-09 04:45:29 on dist-test-slave-1zqn"
server_key: "2d4bccc5a92772d602d1b31fe67807fd"
server_key_iv: "09971f50f10ec21480e0b0899df3274b"
server_key_version: "encryptionkey@0"
I20260809 04:45:29.873482 14863 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:29.876012 14880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:29.877346 14863 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:29.877453 14863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "9e91f0d374ee4533b1ee5b6f0f016253"
format_stamp: "Formatted at 2026-08-09 04:45:29 on dist-test-slave-1zqn"
server_key: "2d4bccc5a92772d602d1b31fe67807fd"
server_key_iv: "09971f50f10ec21480e0b0899df3274b"
server_key_version: "encryptionkey@0"
I20260809 04:45:29.877542 14863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:29.901935 14863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:29.905635 14863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:29.905804 14863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:29.906389 14863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:29.907361 14863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:29.907411 14863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:29.907456 14863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:29.907475 14863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:29.917028 14863 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:36007
I20260809 04:45:29.917047 15011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:36007 every 8 connection(s)
I20260809 04:45:29.918005 14863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:29.921190 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 14863
I20260809 04:45:29.921317 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:29.921687 7852 external_mini_cluster.cc:1428] Setting key 0761e6ef830d58fc28fb9935cc522dd7
Aug 09 04:45:29 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250729, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:45:29.933923 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:29.920013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:34903 (local address 127.7.171.62:36799)
0809 04:45:29.920188 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:29.920193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:29.921493 (+ 1300us) server_negotiation.cc:407] Connection header received
0809 04:45:29.922571 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:29.922575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:29.922622 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:29.922713 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:29.924092 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:29.924647 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:29.926340 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:29.926608 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:29.926697 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:29.929598 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:29.929604 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:29.929610 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:29.929647 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:29.931505 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:29.932049 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:29.932069 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:29.932071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:29.932166 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:29.932619 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:29.932625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:29.932628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:29.932886 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:29.932991 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:29.933679 (+ 688us) server_negotiation.cc:299] Negotiation successful
0809 04:45:29.933793 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:45:29.935056 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:29.920479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.1:34903)
0809 04:45:29.921225 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:29.921315 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:29.922317 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:29.922883 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:29.922892 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:29.923359 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:29.923905 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:29.923920 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:29.924793 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:29.924799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:29.926177 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:29.926193 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:29.926786 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:29.926792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:29.927030 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:29.927737 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:29.927756 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:29.929395 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:29.931637 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:29.931642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:29.931655 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:29.931928 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:29.932311 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:29.932314 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:29.932315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:29.932501 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:29.933042 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:29.933048 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:29.933377 (+ 329us) client_negotiation.cc:769] Sending connection context
0809 04:45:29.933633 (+ 256us) client_negotiation.cc:240] Negotiation successful
0809 04:45:29.933919 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.2@KRBTEST.COM" created.
I20260809 04:45:29.936553 15012 heartbeater.cc:347] Connected to a master server at 127.7.171.62:36799
I20260809 04:45:29.936863 15012 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:29.937531 15012 heartbeater.cc:511] Master 127.7.171.62:36799 requested a full tablet report, sending...
I20260809 04:45:29.939229 14790 ts_manager.cc:194] Registered new tserver with Master: 9e91f0d374ee4533b1ee5b6f0f016253 (127.7.171.1:36007)
I20260809 04:45:29.940522 14790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:34903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab.
Entry for principal oryx/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab.
I20260809 04:45:29.956264 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36799
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab
--principal=oryx/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:30.071090 15017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:30.071389 15017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:30.071435 15017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:30.075567 15017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:30.075635 15017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:30.075696 15017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:30.081174 15017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab
--principal=oryx/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:30.082276 15017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:30.083343 15017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:30.093336 15025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:30.093333 15017 server_base.cc:1034] running on GCE node
W20260809 04:45:30.093333 15024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:30.093561 15027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:30.093976 15017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:30.094633 15017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:30.095824 15017 hybrid_clock.cc:648] HybridClock initialized: now 1786250730095805 us; error 29 us; skew 500 ppm
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:30.099157 15017 init.cc:377] Logged in from keytab as oryx/127.7.171.2@KRBTEST.COM (short username oryx)
I20260809 04:45:30.100152 15017 webserver.cc:460] Webserver started at http://127.7.171.2:40295/ using document root <none> and password file <none>
I20260809 04:45:30.100729 15017 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:30.100822 15017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:30.101068 15017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:30.103332 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "d02b1647f9484f87a92108d3618ff045"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "afff324d44178cbd115df48bc8ac9cb3"
server_key_iv: "c180672927691071a4c09e97749d4e09"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.103909 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "d02b1647f9484f87a92108d3618ff045"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "afff324d44178cbd115df48bc8ac9cb3"
server_key_iv: "c180672927691071a4c09e97749d4e09"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.107677 15017 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:45:30.110306 15036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:30.111573 15017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:30.111724 15017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "d02b1647f9484f87a92108d3618ff045"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "afff324d44178cbd115df48bc8ac9cb3"
server_key_iv: "c180672927691071a4c09e97749d4e09"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.111840 15017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:30.131098 15017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:30.134246 15017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:30.134449 15017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:30.135058 15017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:30.136188 15017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:30.136255 15017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:30.136322 15017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:30.136358 15017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:30.145977 15017 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:37135
I20260809 04:45:30.146000 15163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:37135 every 8 connection(s)
I20260809 04:45:30.146899 15017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:30.154121 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 15017
I20260809 04:45:30.154247 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:30.154623 7852 external_mini_cluster.cc:1428] Setting key 85d518676e3da6973b77dea1e286b699
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:30.162220 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.148849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:58967 (local address 127.7.171.62:36799)
0809 04:45:30.149004 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:30.149010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:30.150170 (+ 1160us) server_negotiation.cc:407] Connection header received
0809 04:45:30.151015 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:30.151017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:30.151062 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:30.151152 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:30.152464 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.152992 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.154495 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.154775 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.154858 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.157647 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:30.157656 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:30.157659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:30.157682 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:30.159598 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.160206 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.160212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.160217 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.160318 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.160680 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.160686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.160691 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.160991 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:30.161119 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:30.161822 (+ 703us) server_negotiation.cc:299] Negotiation successful
0809 04:45:30.161969 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:45:30.163053 15167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.149301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.2:58967)
0809 04:45:30.150021 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:30.150066 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:30.150831 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:30.151315 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:30.151322 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:30.151715 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:30.152335 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.152349 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.153137 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.153140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.154370 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.154382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.154943 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.154948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.155132 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.155770 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:30.155786 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:30.157466 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:30.159729 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.159734 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.159747 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.160037 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.160418 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.160420 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.160422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.160569 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.161149 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:30.161155 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:30.161507 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 04:45:30.161708 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 04:45:30.162062 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":115,"threads_started":1}
I20260809 04:45:30.164623 15164 heartbeater.cc:347] Connected to a master server at 127.7.171.62:36799
I20260809 04:45:30.164940 15164 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:30.165582 15164 heartbeater.cc:511] Master 127.7.171.62:36799 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:45:30.166828 14790 ts_manager.cc:194] Registered new tserver with Master: d02b1647f9484f87a92108d3618ff045 (127.7.171.2:37135)
I20260809 04:45:30.167444 14790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:58967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab.
Entry for principal oryx/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab.
I20260809 04:45:30.187427 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36799
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab
--principal=oryx/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:30.306202 15170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:30.306536 15170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:30.306618 15170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:30.310745 15170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:30.310860 15170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:30.310959 15170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:30.316591 15170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab
--principal=oryx/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:36799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:30.317723 15170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:30.318842 15170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:30.329082 15175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:30.329082 15176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:30.329082 15179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:30.329340 15170 server_base.cc:1034] running on GCE node
I20260809 04:45:30.329830 15170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:30.330441 15170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:30.331627 15170 hybrid_clock.cc:648] HybridClock initialized: now 1786250730331602 us; error 50 us; skew 500 ppm
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:30.334524 15170 init.cc:377] Logged in from keytab as oryx/127.7.171.3@KRBTEST.COM (short username oryx)
I20260809 04:45:30.335423 15170 webserver.cc:460] Webserver started at http://127.7.171.3:37843/ using document root <none> and password file <none>
I20260809 04:45:30.335950 15170 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:30.336140 15170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:30.336336 15170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:30.338340 15170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "5dc661cd26ea43069548bd55457a4beb"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "e479d65af0fc19352f3fe0e94c082d91"
server_key_iv: "38ffbb57e1bdfe6a572b76a25ded7720"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.338922 15170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "5dc661cd26ea43069548bd55457a4beb"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "e479d65af0fc19352f3fe0e94c082d91"
server_key_iv: "38ffbb57e1bdfe6a572b76a25ded7720"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.342682 15170 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 04:45:30.345172 15186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:30.346403 15170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:30.346527 15170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "5dc661cd26ea43069548bd55457a4beb"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "e479d65af0fc19352f3fe0e94c082d91"
server_key_iv: "38ffbb57e1bdfe6a572b76a25ded7720"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.346642 15170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:30.369838 15170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:30.373157 15170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:30.373374 15170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:30.374017 15170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:30.374970 15170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:30.375020 15170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:30.375085 15170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:30.375118 15170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:30.385207 15170 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:41037
I20260809 04:45:30.385228 15315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:41037 every 8 connection(s)
I20260809 04:45:30.386123 15170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:30.395375 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 15170
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:30.395511 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:30.395813 7852 external_mini_cluster.cc:1428] Setting key ce53fc70dad6331f0515cac3662207bb
I20260809 04:45:30.399984 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.387997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:34397 (local address 127.7.171.62:36799)
0809 04:45:30.388200 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:30.388206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:30.389265 (+ 1059us) server_negotiation.cc:407] Connection header received
0809 04:45:30.390118 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:30.390120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:30.390170 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:30.390246 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:30.391461 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.391985 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.393178 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.393386 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.393467 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.396157 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:30.396160 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:30.396162 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:30.396185 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:30.397762 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.398328 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.398332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.398334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.398420 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.398817 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.398820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.398822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.399008 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:30.399117 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:30.399696 (+ 579us) server_negotiation.cc:299] Negotiation successful
0809 04:45:30.399817 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 04:45:30.400775 15318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.388437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.3:34397)
0809 04:45:30.389115 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:30.389156 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:30.389914 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:30.390415 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:30.390422 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:30.390800 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:30.391332 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.391346 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.392164 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.392166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.393069 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.393078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.393518 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.393520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.393656 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.394281 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:30.394297 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:30.395952 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:30.397882 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.397888 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.397904 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.398218 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.398510 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.398514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.398516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.398709 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.399117 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:30.399125 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:30.399386 (+ 261us) client_negotiation.cc:769] Sending connection context
0809 04:45:30.399638 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 04:45:30.399952 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":115,"threads_started":1}
I20260809 04:45:30.401847 15316 heartbeater.cc:347] Connected to a master server at 127.7.171.62:36799
I20260809 04:45:30.402107 15316 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:30.402609 15316 heartbeater.cc:511] Master 127.7.171.62:36799 requested a full tablet report, sending...
I20260809 04:45:30.403602 14790 ts_manager.cc:194] Registered new tserver with Master: 5dc661cd26ea43069548bd55457a4beb (127.7.171.3:41037)
I20260809 04:45:30.404273 14790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:34397
I20260809 04:45:30.410022 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.171.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.61.keytab.
Entry for principal oryx/127.7.171.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.61.keytab.
I20260809 04:45:30.441377 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.61:33893
--webserver_interface=127.7.171.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.61.keytab
--principal=oryx/127.7.171.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:36799,127.7.171.61:33893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:30.569839 15322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:30.570159 15322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:30.570242 15322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:30.576025 15322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:30.576129 15322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:30.576161 15322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:30.576191 15322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:30.576225 15322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:30.582556 15322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:36799,127.7.171.61:33893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.61.keytab
--principal=oryx/127.7.171.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.61:33893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.171.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
W20260809 04:45:30.582813 15322 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.171.62:36799,127.7.171.61:33893'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 04:45:30.583793 15322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:30.584952 15322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:45:30.594007 15322 server_base.cc:1034] running on GCE node
W20260809 04:45:30.593982 15328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:30.594065 15327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:30.593982 15330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:30.594810 15322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:30.596005 15322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:30.597227 15322 hybrid_clock.cc:648] HybridClock initialized: now 1786250730597203 us; error 40 us; skew 500 ppm
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:30.600487 15322 init.cc:377] Logged in from keytab as oryx/127.7.171.61@KRBTEST.COM (short username oryx)
I20260809 04:45:30.601482 15322 webserver.cc:460] Webserver started at http://127.7.171.61:39871/ using document root <none> and password file <none>
I20260809 04:45:30.602123 15322 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:30.602191 15322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:30.602460 15322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:30.604969 15322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data/instance:
uuid: "6570f8e34ec04ce8a4eda09c02115105"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "4e168c77623301a48eef33f4e7c40ff7"
server_key_iv: "5484d030ebbde8f268e2527c1468a65b"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.605571 15322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/wal/instance:
uuid: "6570f8e34ec04ce8a4eda09c02115105"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "4e168c77623301a48eef33f4e7c40ff7"
server_key_iv: "5484d030ebbde8f268e2527c1468a65b"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.609627 15322 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:30.612736 15338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:30.614216 15322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:45:30.614398 15322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/wal
uuid: "6570f8e34ec04ce8a4eda09c02115105"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "4e168c77623301a48eef33f4e7c40ff7"
server_key_iv: "5484d030ebbde8f268e2527c1468a65b"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.614544 15322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:30.645211 15322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:30.648618 15322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:30.648864 15322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:30.656859 15322 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.61:33893
I20260809 04:45:30.657192 15397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.61:33893 every 8 connection(s)
I20260809 04:45:30.657948 15322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
I20260809 04:45:30.659534 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 15322
I20260809 04:45:30.661108 15398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:30.665812 15398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.171.61@KRBTEST.COM
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.61@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:30.675922 15401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.661190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.7.171.61:33893)
0809 04:45:30.661661 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:30.661680 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:30.661723 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:45:30.662604 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:30.662626 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:30.663042 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:30.663443 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:30.664516 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.666011 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.666978 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.667609 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.667805 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.670631 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:30.670639 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:30.670651 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:30.670685 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:30.672697 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.673162 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.673168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.673181 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.673294 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.673597 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.673600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.673602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.673986 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:30.674217 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:30.674468 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 04:45:30.674813 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":125,"threads_started":1}
I20260809 04:45:30.677553 15402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.667653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:37988)
0809 04:45:30.668166 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:30.668214 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:30.668372 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:30.668727 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:30.668732 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:30.669076 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:30.669214 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.669220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.670127 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.670131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.671056 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.671066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.671556 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.671561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.671691 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.672208 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:30.672240 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:30.673680 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:30.675667 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.675676 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.675680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.675930 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.676314 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.676318 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.676320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.676430 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.676864 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:30.676870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:30.677106 (+ 236us) client_negotiation.cc:769] Sending connection context
0809 04:45:30.677248 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 04:45:30.677397 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":154,"threads_started":1}
I20260809 04:45:30.677596 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.667741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37988 (local address 127.7.171.62:36799)
0809 04:45:30.667898 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:30.667903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:30.668288 (+ 385us) server_negotiation.cc:407] Connection header received
0809 04:45:30.668481 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:30.668486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:30.668549 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:30.668644 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:30.669369 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.669973 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.671187 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.671429 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.671554 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.673834 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:30.673837 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:30.673839 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:30.673864 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:30.675539 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.676088 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.676092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.676094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.676193 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.676534 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.676536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.676537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.676753 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:30.676834 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:30.677330 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 04:45:30.677453 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 04:45:30.679070 15398 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.61@KRBTEST.COM for oryx/127.7.171.61@KRBTEST.COM
I20260809 04:45:30.687512 15402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.679549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:33893 (local address 127.0.0.1:48196)
0809 04:45:30.679660 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:30.679674 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:30.679744 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:30.680000 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:30.680003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:30.680420 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:30.680525 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.680530 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.681655 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.681657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.682178 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.682186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.682658 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.682661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.682729 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.683138 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:30.683150 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:30.684398 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:30.686059 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.686063 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.686065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.686261 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.686538 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.686542 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.686544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.686626 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.686978 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:30.686981 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:30.687058 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 04:45:30.687181 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 04:45:30.687329 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 04:45:30.687512 15401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.679591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.7.171.61:33893)
0809 04:45:30.679705 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:30.679708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:30.679720 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:30.679825 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:30.679827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:30.679859 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:30.679925 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:30.680624 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.681545 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.682281 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.682552 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.682641 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.684526 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:30.684530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:30.684534 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:30.684559 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:30.685933 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.686357 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.686363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.686365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.686449 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.686720 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.686723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.686725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.686884 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:30.686990 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:30.687253 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 04:45:30.687386 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":44}
I20260809 04:45:30.690407 15398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: Bootstrap starting.
I20260809 04:45:30.692768 15398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:30.693487 15398 log.cc:824] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:30.695359 15398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: No bootstrap required, opened a new log
I20260809 04:45:30.698378 15398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } }
I20260809 04:45:30.698590 15398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:30.698627 15398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6570f8e34ec04ce8a4eda09c02115105, State: Initialized, Role: FOLLOWER
I20260809 04:45:30.699026 15398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } }
I20260809 04:45:30.699476 15404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } } }
I20260809 04:45:30.699608 15404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:45:30.700317 15398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:45:30.705111 15416 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:45:30.705216 15416 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:45:30.707937 15322 master_runner.cc:428] Detected that this master 6570f8e34ec04ce8a4eda09c02115105 is joining an existing cluster
I20260809 04:45:30.708002 15322 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.171.61:33893
I20260809 04:45:30.719614 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.710355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37998 (local address 127.7.171.62:36799)
0809 04:45:30.710499 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:30.710505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:30.710751 (+ 246us) server_negotiation.cc:407] Connection header received
0809 04:45:30.710925 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:30.710928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:30.710985 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:30.711079 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:30.711673 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.712544 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.713835 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.714213 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.714358 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.715999 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:30.716003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:30.716006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:30.716040 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:30.717668 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.718199 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.718204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.718206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.718294 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.718608 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.718612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.718614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.718848 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:30.718950 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:30.719270 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 04:45:30.719398 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:45:30.720118 15401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.710719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.7.171.61:33893)
0809 04:45:30.710818 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:30.710822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:30.710836 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:30.711205 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:30.711208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:30.711249 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:30.711331 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:30.711819 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.712373 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.714503 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.714851 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.714959 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.716209 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:30.716212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:30.716215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:30.716237 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:30.717956 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.718429 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.718435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.718438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.718501 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.719204 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.719207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.719208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.719406 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:30.719498 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:30.719782 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 04:45:30.719893 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":37}
I20260809 04:45:30.720168 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.710304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:37998)
0809 04:45:30.710658 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:30.710673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:30.710822 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:30.711106 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:30.711108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:30.711327 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:30.711542 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.711549 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.712664 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.712667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.713703 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.713715 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.714353 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.714356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.714421 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.715313 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:30.715330 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:30.715855 (+ 525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:30.717802 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.717805 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.717807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.718093 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.718388 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.718392 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.718394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.718498 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.718958 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:30.718961 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:30.719061 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 04:45:30.719845 (+ 784us) client_negotiation.cc:240] Negotiation successful
0809 04:45:30.719967 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":77,"spinlock_wait_cycles":11008,"thread_start_us":111,"threads_started":1}
I20260809 04:45:30.720604 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.710363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:33893 (local address 127.0.0.1:48198)
0809 04:45:30.710690 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:30.710705 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:30.710822 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:30.711369 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:30.711372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:30.711599 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:30.711716 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.711721 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.712488 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.712494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.713704 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.713715 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.714964 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.714971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.715097 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.715554 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:30.715576 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:30.716046 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:30.718087 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.718090 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.718092 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.718319 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.718997 (+ 678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.719001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.719003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.719092 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.719532 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:30.719535 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:30.719609 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 04:45:30.720306 (+ 697us) client_negotiation.cc:240] Negotiation successful
0809 04:45:30.720433 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":107,"spinlock_wait_cycles":384,"thread_start_us":107,"threads_started":1}
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250729, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.62@KRBTEST.COM for oryx/127.7.171.61@KRBTEST.COM
I20260809 04:45:30.734442 15401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.725133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.7.171.61:33893)
0809 04:45:30.725251 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:30.725255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:30.725612 (+ 357us) server_negotiation.cc:407] Connection header received
0809 04:45:30.725809 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:30.725812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:30.725862 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:30.725943 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:30.726541 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.727290 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.728185 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.728516 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.728628 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.730974 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:30.730978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:30.730981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:30.731009 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:30.732445 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.733042 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.733045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.733047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.733138 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.733455 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.733458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.733460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.733710 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:30.733826 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:30.734250 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 04:45:30.734337 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:45:30.734555 15428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.725092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:33893 (local address 127.0.0.1:48202)
0809 04:45:30.725497 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:30.725549 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:30.725677 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:30.725985 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:30.725990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:30.726309 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:30.726411 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.726416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.727394 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.727397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.728035 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.728043 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.728638 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.728642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.728771 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.729317 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:30.729331 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:30.730839 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:30.732567 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.732580 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.732588 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.732914 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.733243 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.733248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.733251 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.733355 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.733889 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:30.733896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:30.734026 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 04:45:30.734180 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 04:45:30.734380 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":123,"threads_started":1}
I20260809 04:45:30.736102 14790 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.171.61:33893: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105"
member_type: NON_VOTER
last_known_addr {
host: "127.7.171.61"
port: 33893
}
attrs {
promote: true
}
}
dest_uuid: "a936256175c245da8101aab12ceb2c23"
cas_config_opid_index: -1
I20260809 04:45:30.736850 14790 consensus_queue.cc:237] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } }
I20260809 04:45:30.738785 15432 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6570f8e34ec04ce8a4eda09c02115105 (127.7.171.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } }
I20260809 04:45:30.739425 15368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:30.740595 15432 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.171.61:33893
I20260809 04:45:30.740854 15368 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 FOLLOWER]: Refusing update from remote peer a936256175c245da8101aab12ceb2c23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 04:45:30.741024 15322 master.cc:537] Master@127.7.171.61:33893 shutting down...
WARNING: no policy specified for oryx/127.7.171.60@KRBTEST.COM; defaulting to no policy
I20260809 04:45:30.743078 15432 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:30.743505 15434 sys_catalog.cc:455] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } }
I20260809 04:45:30.743652 15434 sys_catalog.cc:458] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.60@KRBTEST.COM" created.
W20260809 04:45:30.745790 14780 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 04:45:30.746711 14780 consensus_peers.cc:487] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 -> Peer 6570f8e34ec04ce8a4eda09c02115105 (127.7.171.61:33893): Couldn't send request to peer 6570f8e34ec04ce8a4eda09c02115105. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 04:45:30.748034 15322 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:30.748342 15322 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:30.748445 15322 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: stopping tablet replica
I20260809 04:45:30.754030 15322 master.cc:552] Master@127.7.171.61:33893 shutdown complete.
I20260809 04:45:30.754083 15322 master_runner.cc:305] Clearing existing system tablet
I20260809 04:45:30.754329 15322 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:30.754388 15322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:30.755676 15322 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 04:45:30.757021 15440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:30.757279 15322 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:30.757433 15322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 04:45:30.757507 15322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/wal
uuid: "6570f8e34ec04ce8a4eda09c02115105"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "4e168c77623301a48eef33f4e7c40ff7"
server_key_iv: "5484d030ebbde8f268e2527c1468a65b"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.759527 15322 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:30.762768 15322 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 04:45:30.762914 15322 log.cc:1196] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 04:45:30.763238 15322 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: Deleting consensus metadata
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.60.keytab.
Entry for principal oryx/127.7.171.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.60.keytab.
I20260809 04:45:30.763450 15322 master_runner.cc:315] Copying system tablet from 127.7.171.62:36799
I20260809 04:45:30.764652 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.60:43709
--webserver_interface=127.7.171.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.60.keytab
--principal=oryx/127.7.171.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:36799,127.7.171.61:33893,127.7.171.60:43709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:45:30.765038 15322 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: tablet copy: Beginning tablet copy session from remote peer at address 127.7.171.62:36799
I20260809 04:45:30.777172 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.765816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38014)
0809 04:45:30.766228 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:30.766244 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:30.766362 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:30.767594 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:30.767597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:30.767830 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:30.767968 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.767975 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.769340 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.769344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.770334 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.770346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.771140 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.771144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:30.771248 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.771921 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:30.771937 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:30.772501 (+ 564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:30.775260 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.775264 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.775266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.775527 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.775899 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:30.775902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:30.775905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:30.775999 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:30.776574 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:30.776578 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:30.776689 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 04:45:30.776792 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 04:45:30.776945 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":226,"threads_started":1}
I20260809 04:45:30.777693 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.767088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38014 (local address 127.7.171.62:36799)
0809 04:45:30.767264 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:30.767271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:30.767291 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:45:30.767352 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:30.767356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:30.767432 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:30.767546 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:30.768263 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.769181 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.770508 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.770985 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.771141 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:30.772704 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:30.772711 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:30.772716 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:30.772770 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:30.775105 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.775661 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.775668 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.775671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.775785 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:30.776136 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:30.776143 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:30.776147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:30.776445 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:30.776582 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:30.776926 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 04:45:30.777103 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:45:30.778720 14818 tablet_copy_service.cc:140] P a936256175c245da8101aab12ceb2c23: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6570f8e34ec04ce8a4eda09c02115105 ({username='oryx', principal='oryx/127.7.171.61@KRBTEST.COM'} at 127.0.0.1:38014)
I20260809 04:45:30.778925 14818 tablet_copy_service.cc:161] P a936256175c245da8101aab12ceb2c23: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6570f8e34ec04ce8a4eda09c02115105 at {username='oryx', principal='oryx/127.7.171.61@KRBTEST.COM'} at 127.0.0.1:38014: session id = 6570f8e34ec04ce8a4eda09c02115105-00000000000000000000000000000000
I20260809 04:45:30.781088 14818 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 04:45:30.783203 15322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 04:45:30.786019 15322 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: tablet copy: Starting download of 0 data blocks...
I20260809 04:45:30.786295 15322 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: tablet copy: Starting download of 1 WAL segments...
I20260809 04:45:30.788506 15322 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 04:45:30.790657 14818 tablet_copy_service.cc:342] P a936256175c245da8101aab12ceb2c23: Request end of tablet copy session 6570f8e34ec04ce8a4eda09c02115105-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.171.61@KRBTEST.COM'} at 127.0.0.1:38014
I20260809 04:45:30.790791 14818 tablet_copy_service.cc:433] P a936256175c245da8101aab12ceb2c23: ending tablet copy session 6570f8e34ec04ce8a4eda09c02115105-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6570f8e34ec04ce8a4eda09c02115105
I20260809 04:45:30.799633 15322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:30.801767 15449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:30.805243 15453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:30.805549 15322 server_base.cc:1034] running on GCE node
W20260809 04:45:30.805202 15450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:30.805893 15322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:30.806208 15322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:30.807353 15322 hybrid_clock.cc:648] HybridClock initialized: now 1786250730807339 us; error 36 us; skew 500 ppm
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:30.809273 15322 init.cc:377] Logged in from keytab as oryx/127.7.171.61@KRBTEST.COM (short username oryx)
I20260809 04:45:30.809753 15322 webserver.cc:460] Webserver started at http://127.7.171.61:38893/ using document root <none> and password file <none>
I20260809 04:45:30.809969 15322 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:30.810012 15322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:30.811435 15322 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 04:45:30.812569 15460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:30.812893 15322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 04:45:30.812976 15322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/wal
uuid: "6570f8e34ec04ce8a4eda09c02115105"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "4e168c77623301a48eef33f4e7c40ff7"
server_key_iv: "5484d030ebbde8f268e2527c1468a65b"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.813063 15322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:30.850255 15322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:30.852744 15322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:30.858500 15322 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.61:33893
I20260809 04:45:30.858542 15521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.61:33893 every 8 connection(s)
I20260809 04:45:30.859210 15322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
I20260809 04:45:30.861243 15522 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:45:30.862453 15522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: Bootstrap starting.
I20260809 04:45:30.875554 15522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:45:30.875916 15522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: Bootstrap complete.
I20260809 04:45:30.876607 15522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } }
I20260809 04:45:30.876765 15522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6570f8e34ec04ce8a4eda09c02115105, State: Initialized, Role: LEARNER
I20260809 04:45:30.876876 15522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } }
I20260809 04:45:30.877283 15526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } }
I20260809 04:45:30.877408 15522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:30.877532 15526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:45:30.881428 15540 catalog_manager.cc:1259] Loaded cluster ID: 98ab283728104e609f7f6e43a857e25c
I20260809 04:45:30.881489 15540 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: loading cluster ID for follower catalog manager: success
I20260809 04:45:30.883563 15540 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: acquiring CA information for follower catalog manager: success
I20260809 04:45:30.884959 15540 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 04:45:30.907069 15448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:30.907392 15448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:30.907510 15448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:30.911590 15448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:30.911696 15448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:30.911738 15448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:30.911805 15448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:30.911847 15448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:30.917876 15448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:36799,127.7.171.61:33893,127.7.171.60:43709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.60.keytab
--principal=oryx/127.7.171.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.60:43709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.171.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:30.918994 15448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:30.920147 15448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:30.928781 15547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:30.928787 15549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:30.928802 15546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:30.928921 15448 server_base.cc:1034] running on GCE node
I20260809 04:45:30.929679 15448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:30.930816 15448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:30.932013 15448 hybrid_clock.cc:648] HybridClock initialized: now 1786250730932004 us; error 35 us; skew 500 ppm
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:30.934901 15448 init.cc:377] Logged in from keytab as oryx/127.7.171.60@KRBTEST.COM (short username oryx)
I20260809 04:45:30.935813 15448 webserver.cc:460] Webserver started at http://127.7.171.60:42731/ using document root <none> and password file <none>
I20260809 04:45:30.936450 15448 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:30.936540 15448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:30.936795 15448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:30.938850 15448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data/instance:
uuid: "60952733ca774706ad7ab46575a36282"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "f463e07175236882548b6b440c1541a3"
server_key_iv: "beba1ed25402142b770af29e2c21fe19"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.939430 15448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/wal/instance:
uuid: "60952733ca774706ad7ab46575a36282"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "f463e07175236882548b6b440c1541a3"
server_key_iv: "beba1ed25402142b770af29e2c21fe19"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.943189 15448 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 04:45:30.943572 15012 heartbeater.cc:503] Master 127.7.171.62:36799 was elected leader, sending a full tablet report...
I20260809 04:45:30.946045 15559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:30.947333 15448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:30.947479 15448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/wal
uuid: "60952733ca774706ad7ab46575a36282"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "f463e07175236882548b6b440c1541a3"
server_key_iv: "beba1ed25402142b770af29e2c21fe19"
server_key_version: "encryptionkey@0"
I20260809 04:45:30.947603 15448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:30.972749 15448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:30.976168 15448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:30.976416 15448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:30.984531 15448 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.60:43709
I20260809 04:45:30.984910 15623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.60:43709 every 8 connection(s)
I20260809 04:45:30.985581 15448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
I20260809 04:45:30.988579 15625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:30.992739 15625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:45:30.994632 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 15448
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.60@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
Aug 09 04:45:30 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.171.60@KRBTEST.COM
I20260809 04:45:31.008749 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.994889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.7.171.62:36799)
0809 04:45:30.995079 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:30.995085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:30.996048 (+ 963us) server_negotiation.cc:407] Connection header received
0809 04:45:30.997096 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:30.997100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:30.997156 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:30.997267 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:30.998418 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.999358 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.000866 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.001096 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.001186 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.004003 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.004008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.004013 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.004071 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.005900 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.006533 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.006536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.006537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.006624 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.007370 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.007374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.007376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.007681 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.007761 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.008392 (+ 631us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.008570 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:45:31.009933 15627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.995312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38022)
0809 04:45:30.995889 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:30.995947 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:30.996869 (+ 922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:30.997336 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:30.997342 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:30.997847 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:30.998277 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:30.998291 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.999503 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:30.999506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.000727 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.000738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.001250 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.001254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.001501 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.002269 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.002284 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.003844 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.006008 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.006015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.006027 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.006431 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.007061 (+ 630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.007065 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.007066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.007227 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.007784 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.007790 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.008136 (+ 346us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.008322 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.008923 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":323,"spinlock_wait_cycles":5120,"thread_start_us":172,"threads_started":1}
I20260809 04:45:31.009933 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:30.995643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34666 (local address 127.7.171.60:43709)
0809 04:45:30.996097 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:30.996109 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:30.996150 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:45:30.996851 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:30.996868 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:30.997131 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:30.997318 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:30.998252 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:30.999656 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.000848 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.001287 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.001501 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.004606 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.004611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.004622 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.004643 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.006624 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.007051 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.007057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.007059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.007208 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.007584 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.007586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.007587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.008098 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.008311 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.008522 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.008721 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":327,"thread_start_us":128,"threads_started":1}
I20260809 04:45:31.011724 15625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.60@KRBTEST.COM for oryx/127.7.171.61@KRBTEST.COM
I20260809 04:45:31.020768 15627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.012277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:33893 (local address 127.0.0.1:48204)
0809 04:45:31.012403 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.012421 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.012497 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.012946 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.012949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.013115 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.013213 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.013218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.014231 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.014234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.014878 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.014887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.015417 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.015420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.015495 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.015960 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.015971 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.017327 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.019187 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.019192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.019194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.019397 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.019694 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.019698 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.019700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.019784 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.020234 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.020237 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.020340 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.020464 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.020604 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 04:45:31.020799 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.012404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.7.171.61:33893)
0809 04:45:31.012729 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.012740 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.012753 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:45:31.012789 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.012791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.012830 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.012905 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.013320 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.014109 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.015036 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.015306 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.015399 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.017476 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.017479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.017481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.017504 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.019069 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.019520 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.019523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.019524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.019599 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.019875 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.019879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.019881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.020131 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.020245 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.020520 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.020641 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":116,"threads_started":1}
I20260809 04:45:31.021672 15625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250730, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.60@KRBTEST.COM for oryx/127.7.171.60@KRBTEST.COM
I20260809 04:45:31.029482 15627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.022122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:43709 (local address 127.0.0.1:34676)
0809 04:45:31.022239 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.022254 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.022330 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.022513 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.022516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.022643 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.022712 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.022716 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.023767 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.023769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.024350 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.024358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.024800 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.024803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.024858 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.025233 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.025243 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.026442 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.028167 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.028169 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.028170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.028342 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.028581 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.028583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.028584 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.028652 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.029122 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.029124 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.029194 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.029272 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.029366 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 04:45:31.029583 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.022174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34676 (local address 127.7.171.60:43709)
0809 04:45:31.022285 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.022289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.022303 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:31.022388 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.022390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.022422 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.022486 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.022804 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.023666 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.024461 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.024700 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.024789 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.026563 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.026565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.026567 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.026592 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.028026 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.028424 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.028427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.028429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.028501 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.028766 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.028768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.028770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.029022 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.029125 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.029334 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.029438 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:45:31.032774 15625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: Bootstrap starting.
I20260809 04:45:31.035370 15625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:31.036298 15625 log.cc:824] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:31.038341 15625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: No bootstrap required, opened a new log
I20260809 04:45:31.041453 15625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } }
I20260809 04:45:31.041649 15625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:31.041682 15625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60952733ca774706ad7ab46575a36282, State: Initialized, Role: FOLLOWER
I20260809 04:45:31.042097 15625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } }
I20260809 04:45:31.042519 15633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } } }
I20260809 04:45:31.042673 15633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:45:31.043231 15625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:45:31.048115 15646 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:45:31.048195 15646 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:45:31.052299 15448 master_runner.cc:428] Detected that this master 60952733ca774706ad7ab46575a36282 is joining an existing cluster
I20260809 04:45:31.052350 15448 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.171.60:43709
I20260809 04:45:31.063840 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.055365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.7.171.60:43709)
0809 04:45:31.055452 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.055456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.055472 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:45:31.055498 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.055499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.055527 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.055617 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.056461 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.057166 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.058704 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.059021 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.059100 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.060588 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.060591 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.060594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.060616 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.062093 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.062531 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.062537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.062539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.062618 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.062888 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.062894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.062896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.063126 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.063217 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.063560 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.063696 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":31}
I20260809 04:45:31.063851 15655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.054890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:43709 (local address 127.0.0.1:34686)
0809 04:45:31.055232 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.055246 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.055318 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.055627 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.055629 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.056227 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.056341 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.056348 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.057284 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.057287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.058563 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.058573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.059252 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.059257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.059374 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.059959 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.059971 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.060440 (+ 469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.062219 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.062222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.062225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.062420 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.062715 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.062718 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.062720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.062803 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.063251 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.063254 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.063334 (+ 80us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.063497 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.063642 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":231,"thread_start_us":82,"threads_started":1}
I20260809 04:45:31.063871 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 14863
I20260809 04:45:31.075085 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:36007
--local_ip_for_outbound_sockets=127.7.171.1
--tserver_master_addrs=127.7.171.62:36799
--webserver_port=34817
--webserver_interface=127.7.171.1
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab
--principal=oryx/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.171.62:36799,127.7.171.61:33893
--tserver_master_addrs=127.7.171.62:36799,127.7.171.61:33893,127.7.171.60:43709 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:45:31.081887 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.055264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.7.171.61:33893)
0809 04:45:31.055497 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.055502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.055517 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:31.055564 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.055567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.055616 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.055702 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.056375 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.057187 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.058715 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.059051 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.059138 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.077700 (+ 18562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.077705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.077709 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.077743 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.079723 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.080349 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.080353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.080355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.080460 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.080801 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.080805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.080807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.081059 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.081152 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.081539 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.081693 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260809 04:45:31.082182 15654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.054445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38028)
0809 04:45:31.054771 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.054787 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.054937 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.055355 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.055361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.055951 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.056141 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.056149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.057284 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.057287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.058563 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.058573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.059276 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.059279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.059467 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.076718 (+ 17251us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.076744 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.077372 (+ 628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.080162 (+ 2790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.080168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.080173 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.080453 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.080824 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.080830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.080835 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.080937 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.081507 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.081511 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.081643 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.081754 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.081944 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":235,"spinlock_wait_cycles":3840,"thread_start_us":91,"threads_started":1}
I20260809 04:45:31.082510 15653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.054459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:33893 (local address 127.0.0.1:48218)
0809 04:45:31.054829 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.054843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.054937 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.055826 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.055829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.056090 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.056226 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.056234 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.057367 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.057370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.058559 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.058569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.059295 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.059299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.059472 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.076931 (+ 17459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.076951 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.077456 (+ 505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.079888 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.079892 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.079895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.080215 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.080593 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.080597 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.080599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.080692 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.081215 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.081218 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.081326 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.082134 (+ 808us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.082297 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":169,"spinlock_wait_cycles":4608,"thread_start_us":120,"threads_started":1}
I20260809 04:45:31.082831 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.054422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38028 (local address 127.7.171.62:36799)
0809 04:45:31.054571 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.054576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.054860 (+ 284us) server_negotiation.cc:407] Connection header received
0809 04:45:31.055075 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.055079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.055138 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.055214 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.056285 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.057153 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.058713 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.059051 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.059117 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.077633 (+ 18516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.077639 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.077643 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.077681 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.079955 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.080597 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.080604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.080608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.080706 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.081056 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.081062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.081066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.081347 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.081453 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.082532 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.082682 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250729, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.62@KRBTEST.COM for oryx/127.7.171.60@KRBTEST.COM
I20260809 04:45:31.096887 15428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.087334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:43709 (local address 127.0.0.1:34692)
0809 04:45:31.087528 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.087543 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.087624 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.087965 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.087968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.088205 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.088330 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.088338 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.089469 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.089473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.090336 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.090348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.090962 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.090967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.091078 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.091575 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.091593 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.093239 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.095253 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.095257 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.095259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.095504 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.095786 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.095789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.095791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.095871 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.096350 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.096353 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.096462 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.096558 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.096706 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260809 04:45:31.097380 15628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.087430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.7.171.60:43709)
0809 04:45:31.087569 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.087574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.087590 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:45:31.087753 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.087758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.087830 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.087934 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.088479 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.089336 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.090481 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.090828 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.090934 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.093400 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.093404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.093406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.093431 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.095125 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.095608 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.095612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.095614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.095693 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.095960 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.095963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.095965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.096207 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.096303 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.097116 (+ 813us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.097230 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:45:31.098395 14790 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.171.60:43709: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "60952733ca774706ad7ab46575a36282"
member_type: NON_VOTER
last_known_addr {
host: "127.7.171.60"
port: 43709
}
attrs {
promote: true
}
}
dest_uuid: "a936256175c245da8101aab12ceb2c23"
cas_config_opid_index: 5
I20260809 04:45:31.099066 14790 consensus_queue.cc:237] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } }
I20260809 04:45:31.100231 15433 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 60952733ca774706ad7ab46575a36282 (127.7.171.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.101002 15593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:31.102420 15433 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.171.60:43709
I20260809 04:45:31.102594 15593 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 FOLLOWER]: Refusing update from remote peer a936256175c245da8101aab12ceb2c23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 04:45:31.102620 15433 sys_catalog.cc:455] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.102723 15433 sys_catalog.cc:458] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:31.103364 15432 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:31.104748 15448 master.cc:537] Master@127.7.171.60:43709 shutting down...
I20260809 04:45:31.110859 15428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.100266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:33893 (local address 127.0.0.1:48224)
0809 04:45:31.103382 (+ 3116us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.103398 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.103481 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.106540 (+ 3059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.106544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.106565 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:31.106804 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.106811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.107924 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.107928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.109164 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.109176 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.110407 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.110410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.110518 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.110572 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.110646 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.110722 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3044}
I20260809 04:45:31.111244 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.101167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48224 (local address 127.7.171.61:33893)
0809 04:45:31.104811 (+ 3644us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.104816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.104834 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:45:31.104892 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.104895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.104960 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.105068 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:31.106955 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.107770 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.109314 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.110064 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.110157 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.110306 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.111044 (+ 738us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.111134 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3546}
I20260809 04:45:31.111836 15492 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 LEARNER]: Refusing update from remote peer a936256175c245da8101aab12ceb2c23: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 04:45:31.112335 15432 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 04:45:31.114478 15593 raft_consensus.cc:858] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } }
W20260809 04:45:31.114612 15593 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } }
I20260809 04:45:31.114382 15492 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 60952733ca774706ad7ab46575a36282 (127.7.171.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.116299 15593 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6570f8e34ec04ce8a4eda09c02115105 (127.7.171.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } }
I20260809 04:45:31.117295 15526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a936256175c245da8101aab12ceb2c23. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.117410 15526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:45:31.117499 15526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.117558 15526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:45:31.118438 15633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a936256175c245da8101aab12ceb2c23. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } }
I20260809 04:45:31.118553 15633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 04:45:31.118664 15633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } }
I20260809 04:45:31.118733 15633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 04:45:31.118813 15633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: NON_VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: true } } }
I20260809 04:45:31.118878 15633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 04:45:31.119117 15434 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23: attempting to promote NON_VOTER 6570f8e34ec04ce8a4eda09c02115105 to VOTER
I20260809 04:45:31.119594 15434 consensus_queue.cc:237] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } }
I20260809 04:45:31.121227 15492 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 LEARNER]: Refusing update from remote peer a936256175c245da8101aab12ceb2c23: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 04:45:31.121608 15432 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 04:45:31.121840 14780 consensus_peers.cc:487] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 -> Peer 60952733ca774706ad7ab46575a36282 (127.7.171.60:43709): Couldn't send request to peer 60952733ca774706ad7ab46575a36282. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 04:45:31.123519 15433 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6570f8e34ec04ce8a4eda09c02115105 (127.7.171.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.124162 15448 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260809 04:45:31.124107 15492 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6570f8e34ec04ce8a4eda09c02115105 (127.7.171.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.124378 15448 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260809 04:45:31.124451 15448 tablet_replica.cc:330] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: stopping tablet replica
I20260809 04:45:31.125118 15433 sys_catalog.cc:455] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.125232 15433 sys_catalog.cc:458] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:31.125380 15433 sys_catalog.cc:455] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.125464 15433 sys_catalog.cc:458] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:31.125754 15661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.125972 15661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:45:31.144831 15448 master.cc:552] Master@127.7.171.60:43709 shutdown complete.
I20260809 04:45:31.144922 15448 master_runner.cc:305] Clearing existing system tablet
I20260809 04:45:31.145231 15448 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:31.145298 15448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:31.146715 15448 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:31.147780 15668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:31.147994 15448 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:31.148135 15448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:31.148173 15448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/wal
uuid: "60952733ca774706ad7ab46575a36282"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "f463e07175236882548b6b440c1541a3"
server_key_iv: "beba1ed25402142b770af29e2c21fe19"
server_key_version: "encryptionkey@0"
I20260809 04:45:31.150086 15448 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:31.152941 15448 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 04:45:31.153047 15448 log.cc:1196] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 04:45:31.153396 15448 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: Deleting consensus metadata
I20260809 04:45:31.153584 15448 master_runner.cc:315] Copying system tablet from 127.7.171.62:36799
I20260809 04:45:31.154912 15448 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: tablet copy: Beginning tablet copy session from remote peer at address 127.7.171.62:36799
I20260809 04:45:31.164641 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.155641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38034 (local address 127.7.171.62:36799)
0809 04:45:31.155802 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.155809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.156109 (+ 300us) server_negotiation.cc:407] Connection header received
0809 04:45:31.156255 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.156258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.156317 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.156414 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.157009 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.157818 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.159013 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.159349 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.159458 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.160917 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.160922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.160925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.160953 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.162687 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.163215 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.163219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.163221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.163315 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.163643 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.163646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.163648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.163874 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.163975 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.164347 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.164476 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:45:31.164649 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.155626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38034)
0809 04:45:31.155959 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.155976 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.156137 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.156460 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.156464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.156726 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.156867 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.156876 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.157927 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.157932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.158871 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.158882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.159480 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.159484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.159592 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.160257 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.160274 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.160754 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.162806 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.162810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.162813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.163086 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.163428 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.163431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.163433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.163538 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.163979 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.163982 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.164117 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.164282 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.164450 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":75,"threads_started":1}
I20260809 04:45:31.165566 14818 tablet_copy_service.cc:140] P a936256175c245da8101aab12ceb2c23: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 60952733ca774706ad7ab46575a36282 ({username='oryx', principal='oryx/127.7.171.60@KRBTEST.COM'} at 127.0.0.1:38034)
I20260809 04:45:31.165674 14818 tablet_copy_service.cc:161] P a936256175c245da8101aab12ceb2c23: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 60952733ca774706ad7ab46575a36282 at {username='oryx', principal='oryx/127.7.171.60@KRBTEST.COM'} at 127.0.0.1:38034: session id = 60952733ca774706ad7ab46575a36282-00000000000000000000000000000000
I20260809 04:45:31.166851 14818 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 04:45:31.168340 15448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 04:45:31.169909 15164 heartbeater.cc:503] Master 127.7.171.62:36799 was elected leader, sending a full tablet report...
I20260809 04:45:31.170908 15448 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: tablet copy: Starting download of 0 data blocks...
I20260809 04:45:31.171103 15448 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: tablet copy: Starting download of 1 WAL segments...
I20260809 04:45:31.172901 15448 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 04:45:31.174885 14818 tablet_copy_service.cc:342] P a936256175c245da8101aab12ceb2c23: Request end of tablet copy session 60952733ca774706ad7ab46575a36282-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.171.60@KRBTEST.COM'} at 127.0.0.1:38034
I20260809 04:45:31.174970 14818 tablet_copy_service.cc:433] P a936256175c245da8101aab12ceb2c23: ending tablet copy session 60952733ca774706ad7ab46575a36282-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 60952733ca774706ad7ab46575a36282
I20260809 04:45:31.183138 15448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:31.185119 15677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:31.185192 15678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:31.185422 15680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:31.186030 15448 server_base.cc:1034] running on GCE node
I20260809 04:45:31.186333 15448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:31.186635 15448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:31.187758 15448 hybrid_clock.cc:648] HybridClock initialized: now 1786250731187746 us; error 35 us; skew 500 ppm
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:31.189503 15448 init.cc:377] Logged in from keytab as oryx/127.7.171.60@KRBTEST.COM (short username oryx)
I20260809 04:45:31.189954 15448 webserver.cc:460] Webserver started at http://127.7.171.60:41315/ using document root <none> and password file <none>
I20260809 04:45:31.190156 15448 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:31.190191 15448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:31.191676 15448 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 04:45:31.192864 15688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:31.193156 15448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:31.193269 15448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/wal
uuid: "60952733ca774706ad7ab46575a36282"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "f463e07175236882548b6b440c1541a3"
server_key_iv: "beba1ed25402142b770af29e2c21fe19"
server_key_version: "encryptionkey@0"
I20260809 04:45:31.193418 15448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:31.203713 15448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:31.206169 15448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:31.211445 15448 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.60:43709
I20260809 04:45:31.211478 15747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.60:43709 every 8 connection(s)
I20260809 04:45:31.212328 15448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
I20260809 04:45:31.214663 15748 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:45:31.216109 15748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: Bootstrap starting.
I20260809 04:45:31.225314 15748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:45:31.225665 15748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: Bootstrap complete.
I20260809 04:45:31.226315 15748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } }
I20260809 04:45:31.226495 15748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 60952733ca774706ad7ab46575a36282, State: Initialized, Role: LEARNER
I20260809 04:45:31.226617 15748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } }
I20260809 04:45:31.227012 15751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.227151 15748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:31.227234 15751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:45:31.231210 15762 catalog_manager.cc:1259] Loaded cluster ID: 98ab283728104e609f7f6e43a857e25c
I20260809 04:45:31.231279 15762 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: loading cluster ID for follower catalog manager: success
I20260809 04:45:31.233470 15762 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: acquiring CA information for follower catalog manager: success
I20260809 04:45:31.234908 15762 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 04:45:31.239106 15656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:31.239369 15656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:31.239445 15656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:31.243717 15656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:31.243777 15656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:31.243867 15656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:31.249511 15656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab
--principal=oryx/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:36007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=34817
--tserver_master_addrs=127.7.171.62:36799,127.7.171.61:33893,127.7.171.60:43709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:31.250869 15656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:31.251941 15656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:31.262042 15767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:31.262123 15769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:31.262156 15656 server_base.cc:1034] running on GCE node
W20260809 04:45:31.262076 15771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:31.262871 15656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:31.263737 15656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:31.264909 15656 hybrid_clock.cc:648] HybridClock initialized: now 1786250731264887 us; error 31 us; skew 500 ppm
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:31.268188 15656 init.cc:377] Logged in from keytab as oryx/127.7.171.1@KRBTEST.COM (short username oryx)
I20260809 04:45:31.269134 15656 webserver.cc:460] Webserver started at http://127.7.171.1:34817/ using document root <none> and password file <none>
I20260809 04:45:31.269732 15656 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:31.269868 15656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:31.273888 15656 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:45:31.276005 15778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:31.277339 15656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:31.277519 15656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "9e91f0d374ee4533b1ee5b6f0f016253"
format_stamp: "Formatted at 2026-08-09 04:45:29 on dist-test-slave-1zqn"
server_key: "2d4bccc5a92772d602d1b31fe67807fd"
server_key_iv: "09971f50f10ec21480e0b0899df3274b"
server_key_version: "encryptionkey@0"
I20260809 04:45:31.278189 15656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:31.296970 15656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:31.300485 15656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:31.300660 15656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:31.301206 15656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:31.302172 15656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:31.302222 15656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:31.302264 15656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:31.302286 15656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:31.311247 15656 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:36007
I20260809 04:45:31.311324 15908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:36007 every 8 connection(s)
I20260809 04:45:31.312402 15656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:31.320531 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 15656
I20260809 04:45:31.320760 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 15017
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.61@KRBTEST.COM
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.60@KRBTEST.COM
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:31.332041 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.315147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:35793 (local address 127.7.171.62:36799)
0809 04:45:31.315281 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.315286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.315666 (+ 380us) server_negotiation.cc:407] Connection header received
0809 04:45:31.317028 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.317036 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.317111 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.317204 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.320156 (+ 2952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.320824 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.325410 (+ 4586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.325779 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.325897 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.328424 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.328427 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.328430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.328459 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.330250 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.330701 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.330704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.330706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.330783 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.331114 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.331117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.331118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.331315 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.331396 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.331732 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.331854 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:45:31.332134 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.314385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:37857 (local address 127.7.171.61:33893)
0809 04:45:31.314553 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.314558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.315599 (+ 1041us) server_negotiation.cc:407] Connection header received
0809 04:45:31.317141 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.317148 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.317217 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.317300 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.319108 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.319994 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.322113 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.322368 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.322489 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.327162 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.327169 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.327175 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.327211 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.329758 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.330309 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.330315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.330320 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.330419 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.330746 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.330751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.330756 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.330997 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.331080 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.331714 (+ 634us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.331860 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:45:31.332805 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.316187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:51999 (local address 127.7.171.60:43709)
0809 04:45:31.316587 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.316591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.316608 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:45:31.317359 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.317363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.317451 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.317569 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.319049 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.320345 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.322186 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.322654 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.322779 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.327153 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.327157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.327161 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.327199 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.329392 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.330088 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.330093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.330095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.330187 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.330637 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.330641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.330644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.330899 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.330987 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.332535 (+ 1548us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.332660 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":82,"threads_started":1}
I20260809 04:45:31.333043 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:37135
--local_ip_for_outbound_sockets=127.7.171.2
--tserver_master_addrs=127.7.171.62:36799
--webserver_port=40295
--webserver_interface=127.7.171.2
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab
--principal=oryx/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.171.62:36799,127.7.171.61:33893
--tserver_master_addrs=127.7.171.62:36799,127.7.171.61:33893,127.7.171.60:43709 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:45:31.333171 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.314869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.1:35793)
0809 04:45:31.315442 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.315495 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.316780 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.317478 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.317488 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.318152 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.318976 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.318988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.324650 (+ 5662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.324656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.325270 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.325280 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.326083 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.326087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.326197 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.326720 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.326735 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.328247 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.330359 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.330361 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.330363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.330593 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.330891 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.330893 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.330895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.331008 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.331451 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.331454 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.331544 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.332269 (+ 725us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.332421 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":141,"spinlock_wait_cycles":29440,"thread_start_us":201,"threads_started":1}
I20260809 04:45:31.333186 15914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.314869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:33893 (local address 127.7.171.1:37857)
0809 04:45:31.315441 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.315494 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.316786 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.317478 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.317488 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.318148 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.318980 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.318988 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.321063 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.321069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.321965 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.321975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.322550 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.322555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.322766 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.323756 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.323768 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.326916 (+ 3148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.329901 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.329904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.329906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.330189 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.330540 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.330545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.330550 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.330643 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.331165 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.331170 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.331424 (+ 254us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.332288 (+ 864us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.332515 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":318,"spinlock_wait_cycles":33408,"thread_start_us":134,"threads_started":1}
I20260809 04:45:31.333469 15915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.316024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:43709 (local address 127.7.171.1:51999)
0809 04:45:31.316412 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.316427 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.316780 (+ 353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.317611 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.317615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.318132 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.318879 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.318895 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.320511 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.320514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.322027 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.322042 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.322779 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.322783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.322874 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.323730 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.323754 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.326916 (+ 3162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.329554 (+ 2638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.329561 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.329578 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.329958 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.330295 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.330301 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.330305 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.330515 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.331043 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.331053 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.331415 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.331638 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.331988 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":105,"spinlock_wait_cycles":30208,"thread_start_us":110,"threads_started":1}
I20260809 04:45:31.334812 15910 heartbeater.cc:347] Connected to a master server at 127.7.171.60:43709
I20260809 04:45:31.335088 15909 heartbeater.cc:347] Connected to a master server at 127.7.171.62:36799
I20260809 04:45:31.335103 15910 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:31.335175 15909 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:31.334813 15911 heartbeater.cc:347] Connected to a master server at 127.7.171.61:33893
I20260809 04:45:31.335541 15911 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:31.335765 15911 heartbeater.cc:511] Master 127.7.171.61:33893 requested a full tablet report, sending...
I20260809 04:45:31.335772 15910 heartbeater.cc:511] Master 127.7.171.60:43709 requested a full tablet report, sending...
I20260809 04:45:31.335779 15909 heartbeater.cc:511] Master 127.7.171.62:36799 requested a full tablet report, sending...
I20260809 04:45:31.337141 14790 ts_manager.cc:194] Re-registered known tserver with Master: 9e91f0d374ee4533b1ee5b6f0f016253 (127.7.171.1:36007)
I20260809 04:45:31.337458 15481 ts_manager.cc:194] Registered new tserver with Master: 9e91f0d374ee4533b1ee5b6f0f016253 (127.7.171.1:36007)
I20260809 04:45:31.337459 15706 ts_manager.cc:194] Registered new tserver with Master: 9e91f0d374ee4533b1ee5b6f0f016253 (127.7.171.1:36007)
I20260809 04:45:31.337899 14790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:35793
I20260809 04:45:31.406266 15316 heartbeater.cc:503] Master 127.7.171.62:36799 was elected leader, sending a full tablet report...
W20260809 04:45:31.457567 15918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:31.457942 15918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:31.458009 15918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:31.462236 15918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:31.462293 15918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:31.462352 15918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:31.467878 15918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab
--principal=oryx/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:37135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=40295
--tserver_master_addrs=127.7.171.62:36799,127.7.171.61:33893,127.7.171.60:43709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:31.469094 15918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:31.470273 15918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:31.480666 15925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:31.480764 15927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:31.480683 15924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:31.484295 15918 server_base.cc:1034] running on GCE node
I20260809 04:45:31.484757 15918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:31.485493 15918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:31.486682 15918 hybrid_clock.cc:648] HybridClock initialized: now 1786250731486649 us; error 46 us; skew 500 ppm
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:31.489691 15918 init.cc:377] Logged in from keytab as oryx/127.7.171.2@KRBTEST.COM (short username oryx)
I20260809 04:45:31.490782 15918 webserver.cc:460] Webserver started at http://127.7.171.2:40295/ using document root <none> and password file <none>
I20260809 04:45:31.491374 15918 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:31.491449 15918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:31.495775 15918 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:31.498030 15935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:31.499394 15918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:31.499528 15918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "d02b1647f9484f87a92108d3618ff045"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "afff324d44178cbd115df48bc8ac9cb3"
server_key_iv: "c180672927691071a4c09e97749d4e09"
server_key_version: "encryptionkey@0"
I20260809 04:45:31.500203 15918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:31.525177 15918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:31.529078 15918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:31.529274 15918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:31.529968 15918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:31.530987 15918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:31.531038 15918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:31.531106 15918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:31.531137 15918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:31.542670 15918 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:37135
I20260809 04:45:31.542701 16069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:37135 every 8 connection(s)
I20260809 04:45:31.543756 15918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:31.550851 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 15918
I20260809 04:45:31.551115 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 15170
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.61@KRBTEST.COM
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.60@KRBTEST.COM
I20260809 04:45:31.569805 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:41037
--local_ip_for_outbound_sockets=127.7.171.3
--tserver_master_addrs=127.7.171.62:36799
--webserver_port=37843
--webserver_interface=127.7.171.3
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab
--principal=oryx/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.171.62:36799,127.7.171.61:33893
--tserver_master_addrs=127.7.171.62:36799,127.7.171.61:33893,127.7.171.60:43709 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:45:31.571244 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.548594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:42077 (local address 127.7.171.61:33893)
0809 04:45:31.548725 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.548731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.550098 (+ 1367us) server_negotiation.cc:407] Connection header received
0809 04:45:31.550295 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.550300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.550386 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.550446 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.558348 (+ 7902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.559257 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.560501 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.560836 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.560929 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.565835 (+ 4906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.565842 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.565849 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.565888 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.568161 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.569002 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.569009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.569013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.569128 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.569598 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.569602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.569603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.569877 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.569980 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.570815 (+ 835us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.570972 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:45:31.572770 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.548336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:49329 (local address 127.7.171.60:43709)
0809 04:45:31.548473 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.548480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.548500 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:45:31.549190 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.549196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.549295 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.549360 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.551143 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.552235 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.560219 (+ 7984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.560758 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.560880 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.565851 (+ 4971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.565859 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.565863 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.565906 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.568410 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.569086 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.569092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.569097 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.569204 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.569616 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.569621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.569625 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.569903 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.570056 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.572323 (+ 2267us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.572522 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:45:31.575490 16075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.548038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:33893 (local address 127.7.171.2:42077)
0809 04:45:31.549997 (+ 1959us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.550014 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.550174 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.550703 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.550708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.551138 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.558175 (+ 7037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.558191 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.559444 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.559449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.560356 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.560370 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.561154 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.561161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.561982 (+ 821us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.562526 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.562571 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.565594 (+ 3023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.568324 (+ 2730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.568335 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.568358 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.568875 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.569250 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.569255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.569260 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.569473 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.570062 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.570084 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.570500 (+ 416us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.571520 (+ 1020us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.574206 (+ 2686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1875,"mutex_wait_us":613,"spinlock_wait_cycles":23552,"thread_start_us":71,"threads_started":1}
I20260809 04:45:31.577104 16072 heartbeater.cc:347] Connected to a master server at 127.7.171.61:33893
I20260809 04:45:31.577394 16072 heartbeater.cc:464] Registering TS with master...
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:31.578050 16072 heartbeater.cc:511] Master 127.7.171.61:33893 requested a full tablet report, sending...
I20260809 04:45:31.579475 15481 ts_manager.cc:194] Registered new tserver with Master: d02b1647f9484f87a92108d3618ff045 (127.7.171.2:37135)
I20260809 04:45:31.582414 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.546989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:48979 (local address 127.7.171.62:36799)
0809 04:45:31.547157 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.547164 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.548946 (+ 1782us) server_negotiation.cc:407] Connection header received
0809 04:45:31.549143 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.549147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.549205 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.549288 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.571975 (+ 22687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.572926 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.574120 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.574432 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.574538 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.578554 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.578558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.578561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.578607 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.580556 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.581009 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.581012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.581014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.581113 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.581420 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.581423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.581425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.581683 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.581795 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.582167 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.582295 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:45:31.584522 16076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.546747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.2:48979)
0809 04:45:31.548845 (+ 2098us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.548857 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.548980 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.549475 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.549483 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.550051 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.571832 (+ 21781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.571841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.573102 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.573108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.573971 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.573981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.574708 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.574712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.574812 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.575374 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.575388 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.578370 (+ 2982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.580683 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.580685 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.580687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.580885 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.581219 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.581222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.581224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.581315 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.581824 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.581828 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.581970 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.582089 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.584383 (+ 2294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2016,"mutex_wait_us":2219,"spinlock_wait_cycles":26752,"thread_start_us":174,"threads_started":1}
I20260809 04:45:31.584537 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.546746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:43709 (local address 127.7.171.2:49329)
0809 04:45:31.547452 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.547523 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.548889 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.549580 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.549585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.550051 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.550901 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.550926 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.557676 (+ 6750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.557683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.559974 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.559991 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.561155 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.561162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.561531 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.562581 (+ 1050us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.562604 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.565595 (+ 2991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.568564 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.568570 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.568576 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.568935 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.569332 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.569337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.569342 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.569496 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.570130 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.570133 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.572078 (+ 1945us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.573010 (+ 932us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.584191 (+ 11181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":3042,"spinlock_wait_cycles":16768,"thread_start_us":699,"threads_started":1}
I20260809 04:45:31.585387 16071 heartbeater.cc:347] Connected to a master server at 127.7.171.60:43709
I20260809 04:45:31.585486 16071 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:31.585729 16071 heartbeater.cc:511] Master 127.7.171.60:43709 requested a full tablet report, sending...
I20260809 04:45:31.585759 16070 heartbeater.cc:347] Connected to a master server at 127.7.171.62:36799
I20260809 04:45:31.585822 16070 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:31.585965 16070 heartbeater.cc:511] Master 127.7.171.62:36799 requested a full tablet report, sending...
I20260809 04:45:31.586565 14790 ts_manager.cc:194] Re-registered known tserver with Master: d02b1647f9484f87a92108d3618ff045 (127.7.171.2:37135)
I20260809 04:45:31.586542 15706 ts_manager.cc:194] Registered new tserver with Master: d02b1647f9484f87a92108d3618ff045 (127.7.171.2:37135)
I20260809 04:45:31.587242 14790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:48979
I20260809 04:45:31.633190 16078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.627912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:43709 (local address 127.0.0.1:34694)
0809 04:45:31.628231 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.628248 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.628368 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.628847 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.628852 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.628885 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:31.629005 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.629012 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.630267 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.630274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.631553 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.631565 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.632743 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.632747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.632845 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.632873 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.632942 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.633019 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":85,"threads_started":1}
I20260809 04:45:31.633234 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.628338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.7.171.60:43709)
0809 04:45:31.628499 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.628506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.628523 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:45:31.628594 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.628599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.628679 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.628803 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:31.629141 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.630095 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.631694 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.632596 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.632732 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.632926 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.632969 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.633054 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:45:31.634233 15719 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 04:45:31.634732 15751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a936256175c245da8101aab12ceb2c23. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.634903 15751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: This master's current role is: LEARNER
W20260809 04:45:31.704886 16077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:31.705171 16077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:31.705253 16077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:31.709337 16077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:31.709426 16077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:31.709520 16077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:31.715019 16077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab
--principal=oryx/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:41037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=37843
--tserver_master_addrs=127.7.171.62:36799,127.7.171.61:33893,127.7.171.60:43709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:31.716176 16077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:31.717255 16077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:31.727339 16086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:31.727337 16085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:31.727506 16077 server_base.cc:1034] running on GCE node
W20260809 04:45:31.727365 16088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:31.728120 16077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:31.728856 16077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:31.730046 16077 hybrid_clock.cc:648] HybridClock initialized: now 1786250731730016 us; error 49 us; skew 500 ppm
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:31.732889 16077 init.cc:377] Logged in from keytab as oryx/127.7.171.3@KRBTEST.COM (short username oryx)
I20260809 04:45:31.733897 16077 webserver.cc:460] Webserver started at http://127.7.171.3:37843/ using document root <none> and password file <none>
I20260809 04:45:31.734419 16077 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:31.734489 16077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:31.738821 16077 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:45:31.740891 16096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:31.742156 16077 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:31.742261 16077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "5dc661cd26ea43069548bd55457a4beb"
format_stamp: "Formatted at 2026-08-09 04:45:30 on dist-test-slave-1zqn"
server_key: "e479d65af0fc19352f3fe0e94c082d91"
server_key_iv: "38ffbb57e1bdfe6a572b76a25ded7720"
server_key_version: "encryptionkey@0"
I20260809 04:45:31.742779 16077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:31.754532 16077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:31.757915 16077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:31.758129 16077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:31.758819 16077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:31.759768 16077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:31.759816 16077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:31.759881 16077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:31.759912 16077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:31.770401 16077 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:41037
I20260809 04:45:31.770517 16229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:41037 every 8 connection(s)
I20260809 04:45:31.771592 16077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:31.779448 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 16077
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.60@KRBTEST.COM
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.61@KRBTEST.COM
Aug 09 04:45:31 dist-test-slave-1zqn krb5kdc[14745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250731, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:31.792001 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.773729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:49409 (local address 127.7.171.60:43709)
0809 04:45:31.773893 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.773897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.775263 (+ 1366us) server_negotiation.cc:407] Connection header received
0809 04:45:31.776983 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.776986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.777040 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.777144 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.778800 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.779509 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.781872 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.782205 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.782296 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.786996 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.787001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.787007 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.787048 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.788963 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.789736 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.789740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.789742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.789834 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.790473 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.790476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.790478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.790683 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.790763 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.791638 (+ 875us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.791773 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:45:31.793924 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.775965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.3:44083)
0809 04:45:31.776483 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.776499 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.776613 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.777299 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.777304 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.777916 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.778597 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.778671 (+ 74us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.779799 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.779805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.781474 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.781486 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.782246 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.782250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.782538 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.783433 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.783460 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.786843 (+ 3383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.791199 (+ 4356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.791202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.791205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.791755 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.792376 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.792384 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.792387 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.792481 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.793291 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.793295 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.793390 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.793482 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.793631 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":420,"spinlock_wait_cycles":18176,"thread_start_us":172,"threads_started":1}
I20260809 04:45:31.794225 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.774530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:43709 (local address 127.7.171.3:49409)
0809 04:45:31.775075 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.775123 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.776590 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.777162 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.777172 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.777916 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.778599 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.778619 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.779727 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.779731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.781731 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.781739 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.782497 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.782501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.782625 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.783491 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.783524 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.786835 (+ 3311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.789167 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.789176 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.789193 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.789604 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.789957 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.789961 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.789963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.790366 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.790835 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.790842 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.791290 (+ 448us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.792215 (+ 925us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.792620 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":302,"spinlock_wait_cycles":530560,"thread_start_us":135,"threads_started":1}
I20260809 04:45:31.795534 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.775515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:33893 (local address 127.7.171.3:55893)
0809 04:45:31.775811 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.775826 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.776613 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.777156 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.777166 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.777884 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:31.778603 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.778645 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.779742 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.779747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.781468 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.781478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.782246 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.782250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.782926 (+ 676us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.783641 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:31.783658 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:31.786349 (+ 2691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:31.789321 (+ 2972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.789324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.789327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.790183 (+ 856us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.792636 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:31.792641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:31.792644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:31.792741 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:31.795009 (+ 2268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:31.795012 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:31.795107 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.795197 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.795353 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":420,"thread_start_us":108,"threads_started":1}
I20260809 04:45:31.795696 16231 heartbeater.cc:347] Connected to a master server at 127.7.171.60:43709
I20260809 04:45:31.795922 16231 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:31.796109 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.776186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:44083 (local address 127.7.171.62:36799)
0809 04:45:31.776324 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.776330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.776626 (+ 296us) server_negotiation.cc:407] Connection header received
0809 04:45:31.777139 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.777142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.777198 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.777283 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.778778 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.779644 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.781647 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.781976 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.782068 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.786994 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.786999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.787001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.787027 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.791062 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.792161 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.792167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.792169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.792260 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.792876 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.792880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.792881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.793126 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.793229 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.795758 (+ 2529us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.795914 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:45:31.796450 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.775656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:55893 (local address 127.7.171.61:33893)
0809 04:45:31.775804 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.775809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.775918 (+ 109us) server_negotiation.cc:407] Connection header received
0809 04:45:31.776829 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.776832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.776893 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.776972 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.778786 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.779590 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.781602 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.781933 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.782061 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.786561 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.786566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.786568 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.786597 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.789194 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.791892 (+ 2698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.791908 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.791911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.792001 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.794611 (+ 2610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.794614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.794616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.794861 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.794963 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.796204 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.796316 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 04:45:31.796531 16231 heartbeater.cc:511] Master 127.7.171.60:43709 requested a full tablet report, sending...
I20260809 04:45:31.796926 16246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.784043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.7.171.61:33893)
0809 04:45:31.785284 (+ 1241us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.785290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.785303 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:45:31.785337 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.785340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.785384 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.785518 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.786470 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.787280 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.788648 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.789185 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.789303 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.791250 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.791254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.791257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.791289 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.793379 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.793919 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.793923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.793925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.794256 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.794821 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.794824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.794826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.795011 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.795120 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.795899 (+ 779us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.796078 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1168,"thread_start_us":78,"threads_started":1}
I20260809 04:45:31.797366 16230 heartbeater.cc:347] Connected to a master server at 127.7.171.62:36799
I20260809 04:45:31.797480 16230 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:31.797660 16230 heartbeater.cc:511] Master 127.7.171.62:36799 requested a full tablet report, sending...
I20260809 04:45:31.798249 16232 heartbeater.cc:347] Connected to a master server at 127.7.171.61:33893
I20260809 04:45:31.798347 16232 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:31.798424 14790 ts_manager.cc:194] Re-registered known tserver with Master: 5dc661cd26ea43069548bd55457a4beb (127.7.171.3:41037)
I20260809 04:45:31.798501 16232 heartbeater.cc:511] Master 127.7.171.61:33893 requested a full tablet report, sending...
I20260809 04:45:31.799135 15706 ts_manager.cc:194] Registered new tserver with Master: 5dc661cd26ea43069548bd55457a4beb (127.7.171.3:41037)
I20260809 04:45:31.799191 14790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:44083
I20260809 04:45:31.799955 15481 ts_manager.cc:194] Registered new tserver with Master: 5dc661cd26ea43069548bd55457a4beb (127.7.171.3:41037)
I20260809 04:45:31.801869 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.784754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.7.171.60:43709)
0809 04:45:31.785004 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.785009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.785349 (+ 340us) server_negotiation.cc:407] Connection header received
0809 04:45:31.785546 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.785553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.785616 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.785681 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.788400 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.789282 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.790473 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.790829 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.790892 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.792708 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.792711 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.792714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.792736 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.794681 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.795485 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.795490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.795492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.795578 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.800950 (+ 5372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.800953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.800955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.801207 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.801338 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.801564 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.801689 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":86,"threads_started":1}
I20260809 04:45:31.804153 16244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.782145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38044 (local address 127.7.171.62:36799)
0809 04:45:31.784876 (+ 2731us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.784881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.784895 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:31.784936 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.784940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.785097 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.786122 (+ 1025us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.796829 (+ 10707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.797537 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.798621 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.798973 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.799073 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.800622 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.800626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.800629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.800653 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.802664 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.803059 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.803062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.803063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.803136 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.803326 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.803328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.803329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.803542 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.803641 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.803879 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.803980 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":2651,"thread_start_us":96,"threads_started":1}
I20260809 04:45:31.807137 14790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:31.810911 14790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:31.813624 15663 sys_catalog.cc:455] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: NON_VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: true } } }
I20260809 04:45:31.813766 15663 sys_catalog.cc:458] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:31.815685 16253 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23: attempting to promote NON_VOTER 60952733ca774706ad7ab46575a36282 to VOTER
I20260809 04:45:31.816171 16253 consensus_queue.cc:237] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } }
I20260809 04:45:31.817325 15719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 LEARNER]: Refusing update from remote peer a936256175c245da8101aab12ceb2c23: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 04:45:31.817373 15492 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 FOLLOWER]: Refusing update from remote peer a936256175c245da8101aab12ceb2c23: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 04:45:31.817699 15663 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 04:45:31.817836 16254 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 04:45:31.819621 16253 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 60952733ca774706ad7ab46575a36282 (127.7.171.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:31.819953 15719 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 60952733ca774706ad7ab46575a36282 (127.7.171.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:31.820012 15492 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 60952733ca774706ad7ab46575a36282 (127.7.171.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:31.821261 16253 sys_catalog.cc:455] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:31.821367 16253 sys_catalog.cc:458] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:31.821543 16253 sys_catalog.cc:455] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:31.821581 15751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:31.821645 16253 sys_catalog.cc:458] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:31.821687 15751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:45:31.821815 16253 sys_catalog.cc:455] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:31.821885 16253 sys_catalog.cc:458] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:31.822171 16265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a936256175c245da8101aab12ceb2c23" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:31.822278 16265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:45:31.833508 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.828237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.7.171.2:37135)
0809 04:45:31.828592 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.828608 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.828693 (+ 85us) server_negotiation.cc:407] Connection header received
0809 04:45:31.828851 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.828857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.829140 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.829331 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:31.829543 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.830651 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.832292 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.832902 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.833050 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.833223 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.833280 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.833374 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":158,"threads_started":1}
I20260809 04:45:31.833492 16078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.828013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:37135 (local address 127.0.0.1:49080)
0809 04:45:31.828620 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.828637 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.828730 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.829304 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.829307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.829328 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:31.829443 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.829449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.830791 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.830797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.832142 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.832154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.833053 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.833056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.833155 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.833178 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.833246 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.833334 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":6}
I20260809 04:45:31.833870 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.828263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:41037 (local address 127.0.0.1:34466)
0809 04:45:31.829243 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.829259 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.829350 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.830011 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.830015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.830034 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:31.830137 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.830142 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.831326 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.831331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.832174 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.832201 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.833298 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.833302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.833382 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.833403 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.833496 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.833569 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":59,"thread_start_us":147,"threads_started":1}
I20260809 04:45:31.833976 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.828698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.7.171.3:41037)
0809 04:45:31.829058 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.829072 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.829571 (+ 499us) server_negotiation.cc:407] Connection header received
0809 04:45:31.829660 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.829665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.829852 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.830026 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:31.830240 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.831223 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.832378 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.833185 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.833302 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.833441 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.833524 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.833810 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":106,"threads_started":1}
I20260809 04:45:31.836906 16267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.828969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36007 (local address 127.0.0.1:55876)
0809 04:45:31.830306 (+ 1337us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.830319 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.830428 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.830878 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.830881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.830898 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:31.831033 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.831039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.832991 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.832994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.835061 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.835072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.836483 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.836488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.836579 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.836605 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.836693 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.836769 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1269,"mutex_wait_us":41,"thread_start_us":100,"threads_started":1}
I20260809 04:45:31.837123 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.829505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55876 (local address 127.7.171.1:36007)
0809 04:45:31.829811 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.829818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.830419 (+ 601us) server_negotiation.cc:407] Connection header received
0809 04:45:31.830534 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.830539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.830728 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.830883 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:31.831152 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.832274 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.835208 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.836104 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.836217 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.836371 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.836752 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.836858 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260809 04:45:31.837212 15982 tablet_service.cc:1467] Processing CreateTablet for tablet 0ef0fc2fe4ed46e8a8d1cb122e27ce13 (DEFAULT_TABLE table=test-table [id=f7e98a0ccbe2431389de6437a1fe6221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:31.837076 16150 tablet_service.cc:1467] Processing CreateTablet for tablet 0ef0fc2fe4ed46e8a8d1cb122e27ce13 (DEFAULT_TABLE table=test-table [id=f7e98a0ccbe2431389de6437a1fe6221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:31.838177 16150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ef0fc2fe4ed46e8a8d1cb122e27ce13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:31.838201 15982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ef0fc2fe4ed46e8a8d1cb122e27ce13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:31.840080 15831 tablet_service.cc:1467] Processing CreateTablet for tablet 0ef0fc2fe4ed46e8a8d1cb122e27ce13 (DEFAULT_TABLE table=test-table [id=f7e98a0ccbe2431389de6437a1fe6221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:31.841228 15831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ef0fc2fe4ed46e8a8d1cb122e27ce13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:31.845391 16274 tablet_bootstrap.cc:492] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb: Bootstrap starting.
I20260809 04:45:31.846797 16275 tablet_bootstrap.cc:492] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045: Bootstrap starting.
I20260809 04:45:31.848325 16274 tablet_bootstrap.cc:654] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:31.849041 16275 tablet_bootstrap.cc:654] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:31.849216 16274 log.cc:824] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:31.849381 16276 tablet_bootstrap.cc:492] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253: Bootstrap starting.
I20260809 04:45:31.849888 16275 log.cc:824] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:31.851713 16276 tablet_bootstrap.cc:654] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:31.851744 16274 tablet_bootstrap.cc:492] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb: No bootstrap required, opened a new log
I20260809 04:45:31.851976 16274 ts_tablet_manager.cc:1397] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 04:45:31.852386 16275 tablet_bootstrap.cc:492] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045: No bootstrap required, opened a new log
I20260809 04:45:31.852612 16275 ts_tablet_manager.cc:1397] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:45:31.852644 16276 log.cc:824] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:31.854722 16276 tablet_bootstrap.cc:492] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253: No bootstrap required, opened a new log
I20260809 04:45:31.854910 16276 ts_tablet_manager.cc:1397] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:45:31.856310 16274 raft_consensus.cc:348] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } } peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } }
I20260809 04:45:31.856568 16274 raft_consensus.cc:374] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:31.856633 16274 raft_consensus.cc:729] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc661cd26ea43069548bd55457a4beb, State: Initialized, Role: FOLLOWER
I20260809 04:45:31.857131 16274 consensus_queue.cc:260] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } } peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } }
I20260809 04:45:31.857367 16275 raft_consensus.cc:348] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } } peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } }
I20260809 04:45:31.857640 16275 raft_consensus.cc:374] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:31.857693 16275 raft_consensus.cc:729] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02b1647f9484f87a92108d3618ff045, State: Initialized, Role: FOLLOWER
I20260809 04:45:31.858220 16274 ts_tablet_manager.cc:1428] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 04:45:31.858150 16275 consensus_queue.cc:260] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } } peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } }
I20260809 04:45:31.858595 16230 heartbeater.cc:503] Master 127.7.171.62:36799 was elected leader, sending a full tablet report...
I20260809 04:45:31.859015 16276 raft_consensus.cc:348] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } } peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } }
I20260809 04:45:31.859263 16276 raft_consensus.cc:374] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:31.859309 16276 raft_consensus.cc:729] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e91f0d374ee4533b1ee5b6f0f016253, State: Initialized, Role: FOLLOWER
I20260809 04:45:31.859732 16276 consensus_queue.cc:260] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } } peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } }
I20260809 04:45:31.860656 16276 ts_tablet_manager.cc:1428] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 04:45:31.860994 15909 heartbeater.cc:503] Master 127.7.171.62:36799 was elected leader, sending a full tablet report...
I20260809 04:45:31.868847 16070 heartbeater.cc:503] Master 127.7.171.62:36799 was elected leader, sending a full tablet report...
I20260809 04:45:31.869127 16275 ts_tablet_manager.cc:1428] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045: Time spent starting tablet: real 0.016s user 0.003s sys 0.004s
I20260809 04:45:31.889727 16282 raft_consensus.cc:482] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:31.890098 16282 raft_consensus.cc:504] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } } peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } }
I20260809 04:45:31.891330 16282 leader_election.cc:288] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dc661cd26ea43069548bd55457a4beb (127.7.171.3:41037), 9e91f0d374ee4533b1ee5b6f0f016253 (127.7.171.1:36007)
I20260809 04:45:31.895543 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.891748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:42521 (local address 127.7.171.3:41037)
0809 04:45:31.891985 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.891989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.892001 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:31.892048 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.892102 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.892175 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.892255 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:31.892617 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.893124 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.894466 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.894867 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.894997 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.895063 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.895311 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.895397 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260809 04:45:31.895542 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.891751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:38265 (local address 127.7.171.1:36007)
0809 04:45:31.891921 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.891924 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.891941 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:45:31.891995 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.891998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.892043 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.892184 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:31.892527 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.893153 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.894306 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.894779 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.894993 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.895070 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.895289 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.895378 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:45:31.895629 16076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.891550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:41037 (local address 127.7.171.2:42521)
0809 04:45:31.891671 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.891688 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.891779 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.892298 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.892304 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.892327 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:31.892480 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.892488 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.893236 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.893241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.894315 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.894327 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.895011 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.895014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.895119 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.895198 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.895290 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.895458 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":150}
I20260809 04:45:31.895641 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.891550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36007 (local address 127.7.171.2:38265)
0809 04:45:31.891802 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:31.891817 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:31.891878 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:31.892213 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:31.892219 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:31.892244 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:31.892395 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.892402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.893290 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.893292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.894177 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:31.894186 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.894991 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.894995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:31.895112 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.895164 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 04:45:31.895258 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 04:45:31.895436 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":180}
I20260809 04:45:31.896348 16172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ef0fc2fe4ed46e8a8d1cb122e27ce13" candidate_uuid: "d02b1647f9484f87a92108d3618ff045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc661cd26ea43069548bd55457a4beb" is_pre_election: true
I20260809 04:45:31.896435 15854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ef0fc2fe4ed46e8a8d1cb122e27ce13" candidate_uuid: "d02b1647f9484f87a92108d3618ff045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" is_pre_election: true
I20260809 04:45:31.896687 16172 raft_consensus.cc:2454] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02b1647f9484f87a92108d3618ff045 in term 0.
I20260809 04:45:31.896824 15854 raft_consensus.cc:2454] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02b1647f9484f87a92108d3618ff045 in term 0.
I20260809 04:45:31.897254 15938 leader_election.cc:302] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc661cd26ea43069548bd55457a4beb, d02b1647f9484f87a92108d3618ff045; no voters:
I20260809 04:45:31.897567 16282 raft_consensus.cc:2781] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:31.897656 16282 raft_consensus.cc:482] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:31.897715 16282 raft_consensus.cc:3037] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:31.898904 16282 raft_consensus.cc:504] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } } peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } }
I20260809 04:45:31.899294 16282 leader_election.cc:288] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [CANDIDATE]: Term 1 election: Requested vote from peers 5dc661cd26ea43069548bd55457a4beb (127.7.171.3:41037), 9e91f0d374ee4533b1ee5b6f0f016253 (127.7.171.1:36007)
I20260809 04:45:31.899703 15854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ef0fc2fe4ed46e8a8d1cb122e27ce13" candidate_uuid: "d02b1647f9484f87a92108d3618ff045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e91f0d374ee4533b1ee5b6f0f016253"
I20260809 04:45:31.899701 16172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ef0fc2fe4ed46e8a8d1cb122e27ce13" candidate_uuid: "d02b1647f9484f87a92108d3618ff045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dc661cd26ea43069548bd55457a4beb"
I20260809 04:45:31.899839 15854 raft_consensus.cc:3037] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:31.899840 16172 raft_consensus.cc:3037] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:31.901259 15854 raft_consensus.cc:2454] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02b1647f9484f87a92108d3618ff045 in term 1.
I20260809 04:45:31.901259 16172 raft_consensus.cc:2454] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02b1647f9484f87a92108d3618ff045 in term 1.
I20260809 04:45:31.901598 15936 leader_election.cc:302] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5dc661cd26ea43069548bd55457a4beb, 9e91f0d374ee4533b1ee5b6f0f016253, d02b1647f9484f87a92108d3618ff045; no voters:
I20260809 04:45:31.901837 16282 raft_consensus.cc:2781] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:31.902029 16282 raft_consensus.cc:686] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 1 LEADER]: Becoming Leader. State: Replica: d02b1647f9484f87a92108d3618ff045, State: Running, Role: LEADER
I20260809 04:45:31.902424 16282 consensus_queue.cc:237] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } } peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } }
I20260809 04:45:31.906195 14795 catalog_manager.cc:5495] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 reported cstate change: term changed from 0 to 1, leader changed from <none> to d02b1647f9484f87a92108d3618ff045 (127.7.171.2). New cstate: current_term: 1 leader_uuid: "d02b1647f9484f87a92108d3618ff045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } health_report { overall_health: UNKNOWN } } }
I20260809 04:45:31.934830 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.931287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.7.171.2:37135)
0809 04:45:31.931418 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.931421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.931436 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:31.931517 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.931519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.931578 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.931675 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:31.931889 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.932460 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.933432 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.933637 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.933712 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.933972 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:31.934472 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:31.934569 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.934667 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.934720 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:45:31.940827 16172 raft_consensus.cc:1261] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb [term 1 FOLLOWER]: Refusing update from remote peer d02b1647f9484f87a92108d3618ff045: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:31.940834 15854 raft_consensus.cc:1261] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 1 FOLLOWER]: Refusing update from remote peer d02b1647f9484f87a92108d3618ff045: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:31.941604 16283 consensus_queue.cc:1035] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:31.941799 16282 consensus_queue.cc:1035] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:31.950049 16287 mvcc.cc:205] Tried to move back new op lower bound from 7316482998022422528 to 7316482997875523584. Current Snapshot: MvccSnapshot[applied={T|T < 7316482998022422528}]
I20260809 04:45:31.951534 16289 mvcc.cc:205] Tried to move back new op lower bound from 7316482998022422528 to 7316482997875523584. Current Snapshot: MvccSnapshot[applied={T|T < 7316482998022422528}]
I20260809 04:45:31.953198 16288 mvcc.cc:205] Tried to move back new op lower bound from 7316482998022422528 to 7316482997875523584. Current Snapshot: MvccSnapshot[applied={T|T < 7316482998022422528}]
I20260809 04:45:31.960690 14790 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:31.960840 14790 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:31.965091 14790 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a936256175c245da8101aab12ceb2c23: Sending DeleteTablet for 3 replicas of tablet 0ef0fc2fe4ed46e8a8d1cb122e27ce13
I20260809 04:45:31.965971 15831 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ef0fc2fe4ed46e8a8d1cb122e27ce13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:31 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:55876
I20260809 04:45:31.966012 15982 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ef0fc2fe4ed46e8a8d1cb122e27ce13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:31 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:49080
I20260809 04:45:31.966487 16300 tablet_replica.cc:330] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253: stopping tablet replica
I20260809 04:45:31.966526 16150 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ef0fc2fe4ed46e8a8d1cb122e27ce13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:31 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:34466
I20260809 04:45:31.966823 16300 raft_consensus.cc:2229] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:31.967160 16300 raft_consensus.cc:2258] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:31.967406 16303 tablet_replica.cc:330] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb: stopping tablet replica
I20260809 04:45:31.967644 16302 tablet_replica.cc:330] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045: stopping tablet replica
I20260809 04:45:31.968401 16303 raft_consensus.cc:2229] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:31.968730 16302 raft_consensus.cc:2229] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:31.969019 16300 ts_tablet_manager.cc:1905] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:31.969410 16303 raft_consensus.cc:2258] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:31.969749 16302 raft_consensus.cc:2258] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:31.970110 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 14759
I20260809 04:45:31.973870 16300 ts_tablet_manager.cc:1918] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:31.973999 16300 log.cc:1196] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-0/wal/wals/0ef0fc2fe4ed46e8a8d1cb122e27ce13
I20260809 04:45:31.974401 16300 ts_tablet_manager.cc:1939] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 9e91f0d374ee4533b1ee5b6f0f016253: Deleting consensus metadata
I20260809 04:45:31.975503 16302 ts_tablet_manager.cc:1905] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:31.975711 16303 ts_tablet_manager.cc:1905] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:31.980816 16302 ts_tablet_manager.cc:1918] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:31.980937 16302 log.cc:1196] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/0ef0fc2fe4ed46e8a8d1cb122e27ce13
I20260809 04:45:31.981305 16302 ts_tablet_manager.cc:1939] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P d02b1647f9484f87a92108d3618ff045: Deleting consensus metadata
W20260809 04:45:31.981702 15782 connection.cc:535] server connection from 127.0.0.1:55876 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 04:45:31.982595 16303 ts_tablet_manager.cc:1918] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:31.982712 16303 log.cc:1196] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/0ef0fc2fe4ed46e8a8d1cb122e27ce13
I20260809 04:45:31.983057 16303 ts_tablet_manager.cc:1939] T 0ef0fc2fe4ed46e8a8d1cb122e27ce13 P 5dc661cd26ea43069548bd55457a4beb: Deleting consensus metadata
W20260809 04:45:31.984215 16101 connection.cc:441] server connection from 127.0.0.1:34466 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34466 (request call id 1) could send its response
W20260809 04:45:31.984905 15940 connection.cc:441] server connection from 127.0.0.1:49080 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49080 (request call id 1) could send its response
I20260809 04:45:31.993074 16246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.969322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48230 (local address 127.7.171.61:33893)
0809 04:45:31.969441 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.969446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.983295 (+ 13849us) server_negotiation.cc:407] Connection header received
0809 04:45:31.983471 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.983477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.983541 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.983622 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.984914 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.985681 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.986672 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.987037 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.987161 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.988802 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.988807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.988813 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.988837 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.990627 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.991116 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.991120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.991122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.991206 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.991481 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.991484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.991486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.991684 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.991809 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.992467 (+ 658us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.992593 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:45:31.997687 15630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.988507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48232 (local address 127.7.171.61:33893)
0809 04:45:31.988638 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.988642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.988655 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:45:31.988698 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.988704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.988757 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.988828 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.989781 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.990608 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.991760 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.992158 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.992235 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.994104 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.994109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.994113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.994142 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.995902 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.996389 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.996393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.996395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.996474 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.996766 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.996772 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.996777 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.996980 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.997078 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.997410 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.997522 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:45:31.999182 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.988951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.7.171.60:43709)
0809 04:45:31.989080 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.989084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.989590 (+ 506us) server_negotiation.cc:407] Connection header received
0809 04:45:31.989763 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.989766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.989818 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.989869 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.991094 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.991835 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.993169 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.993508 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.993582 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:31.995178 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:31.995184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:31.995186 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:31.995217 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:31.996953 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.997446 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.997466 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.997468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.997553 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:31.998013 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:31.998017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:31.998019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:31.998234 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:31.998335 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:31.998896 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 04:45:31.999010 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:45:32.003996 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:31.968963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34712 (local address 127.7.171.60:43709)
0809 04:45:31.969087 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:31.969093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:31.994774 (+ 25681us) server_negotiation.cc:407] Connection header received
0809 04:45:31.994988 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:31.994993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:31.995064 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:31.995125 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:31.995939 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.996746 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.998032 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:31.998312 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:31.998374 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:32.000321 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:32.000325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:32.000328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:32.000353 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:32.001911 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:32.002444 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:32.002448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:32.002453 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:32.002654 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:32.003001 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:32.003004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:32.003006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:32.003224 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:32.003329 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:32.003708 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 04:45:32.003835 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":30}
I20260809 04:45:32.955744 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:32.954953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.1:52265)
0809 04:45:32.955486 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:32.955618 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":90,"threads_started":1}
I20260809 04:45:32.956216 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:32.954963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.2:39613)
0809 04:45:32.955488 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:32.956114 (+ 626us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":86,"threads_started":1}
W20260809 04:45:32.956528 16070 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:36799 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
W20260809 04:45:32.956795 15909 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:36799 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
I20260809 04:45:32.957136 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:32.956326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.3:48589)
0809 04:45:32.956928 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:32.957023 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":74,"threads_started":1}
W20260809 04:45:32.957405 16230 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:36799 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
I20260809 04:45:33.465991 16318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936256175c245da8101aab12ceb2c23)
I20260809 04:45:33.466249 16318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } }
I20260809 04:45:33.467267 16318 leader_election.cc:288] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936256175c245da8101aab12ceb2c23 (127.7.171.62:36799), 60952733ca774706ad7ab46575a36282 (127.7.171.60:43709)
I20260809 04:45:33.467965 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.467441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38544)
0809 04:45:33.467753 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.467824 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":22,"thread_start_us":68,"threads_started":1}
I20260809 04:45:33.468011 16321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a936256175c245da8101aab12ceb2c23)
W20260809 04:45:33.468209 15464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
I20260809 04:45:33.468269 16321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } }
I20260809 04:45:33.469275 16321 leader_election.cc:288] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a936256175c245da8101aab12ceb2c23 (127.7.171.62:36799), 6570f8e34ec04ce8a4eda09c02115105 (127.7.171.61:33893)
I20260809 04:45:33.469937 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.469388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38548)
0809 04:45:33.469733 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.469822 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":99,"threads_started":1}
W20260809 04:45:33.470118 15693 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
I20260809 04:45:33.470247 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.469620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38562)
0809 04:45:33.469969 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.470128 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284}
W20260809 04:45:33.470412 15464 leader_election.cc:334] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a936256175c245da8101aab12ceb2c23 (127.7.171.62:36799): Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
I20260809 04:45:33.471040 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.470789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38572)
0809 04:45:33.470900 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.470952 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:45:33.471185 15693 leader_election.cc:334] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a936256175c245da8101aab12ceb2c23 (127.7.171.62:36799): Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
I20260809 04:45:33.472507 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.467441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:43709 (local address 127.0.0.1:34738)
0809 04:45:33.467754 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.467770 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:33.467884 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:33.468492 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:33.468497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:33.468519 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:33.468632 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.468638 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.469640 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.469646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.470948 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.470957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.472166 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.472169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.472243 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.472264 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:45:33.472340 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 04:45:33.472398 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":98,"threads_started":1}
I20260809 04:45:33.472568 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.467810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34738 (local address 127.7.171.60:43709)
0809 04:45:33.468159 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:33.468163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:33.468179 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:45:33.468235 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:33.468238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:33.468289 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:33.468379 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:33.468776 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.469499 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.471093 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.471982 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.472128 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.472191 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:33.472374 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 04:45:33.472449 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":266,"thread_start_us":54,"threads_started":1}
I20260809 04:45:33.473024 15719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6570f8e34ec04ce8a4eda09c02115105" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "60952733ca774706ad7ab46575a36282" is_pre_election: true
I20260809 04:45:33.473248 15719 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6570f8e34ec04ce8a4eda09c02115105 in term 1.
I20260809 04:45:33.473711 15463 leader_election.cc:302] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60952733ca774706ad7ab46575a36282, 6570f8e34ec04ce8a4eda09c02115105; no voters: a936256175c245da8101aab12ceb2c23
I20260809 04:45:33.474061 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.469734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48238 (local address 127.7.171.61:33893)
0809 04:45:33.470057 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:33.470067 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:33.470080 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:45:33.470119 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:33.470122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:33.470172 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:33.470533 (+ 361us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:33.470620 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.471327 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.472944 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.473520 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.473629 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.473691 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:33.473886 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 04:45:33.473962 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":106,"threads_started":1}
I20260809 04:45:33.474061 16325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.469467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:33893 (local address 127.0.0.1:48238)
0809 04:45:33.469756 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.469770 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:33.469851 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:33.470322 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:33.470326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:33.470344 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:33.470428 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.470433 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.471438 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.471442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.472805 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.472813 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.473641 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.473645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.473741 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.473767 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:45:33.473854 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 04:45:33.473930 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":1,"thread_start_us":60,"threads_started":1}
I20260809 04:45:33.474133 16318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 04:45:33.474191 16318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a936256175c245da8101aab12ceb2c23)
I20260809 04:45:33.474239 16318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:45:33.474697 15492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60952733ca774706ad7ab46575a36282" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6570f8e34ec04ce8a4eda09c02115105" is_pre_election: true
I20260809 04:45:33.475499 16318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } }
I20260809 04:45:33.475775 15492 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 60952733ca774706ad7ab46575a36282 in current term 2: Already voted for candidate 6570f8e34ec04ce8a4eda09c02115105 in this term.
I20260809 04:45:33.475979 16318 leader_election.cc:288] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [CANDIDATE]: Term 2 election: Requested vote from peers a936256175c245da8101aab12ceb2c23 (127.7.171.62:36799), 60952733ca774706ad7ab46575a36282 (127.7.171.60:43709)
I20260809 04:45:33.476384 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.476169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38574)
0809 04:45:33.476268 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.476323 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 04:45:33.476365 15719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6570f8e34ec04ce8a4eda09c02115105" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "60952733ca774706ad7ab46575a36282"
I20260809 04:45:33.476491 15719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:45:33.476462 15691 leader_election.cc:302] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60952733ca774706ad7ab46575a36282; no voters: 6570f8e34ec04ce8a4eda09c02115105, a936256175c245da8101aab12ceb2c23
I20260809 04:45:33.477577 15719 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6570f8e34ec04ce8a4eda09c02115105 in term 2.
I20260809 04:45:33.477710 16321 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:45:33.477932 15463 leader_election.cc:302] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60952733ca774706ad7ab46575a36282, 6570f8e34ec04ce8a4eda09c02115105; no voters:
I20260809 04:45:33.478188 16318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:45:33.478426 16318 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [term 2 LEADER]: Becoming Leader. State: Replica: 6570f8e34ec04ce8a4eda09c02115105, State: Running, Role: LEADER
I20260809 04:45:33.478543 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.478278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38576)
0809 04:45:33.478429 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.478484 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260809 04:45:33.478621 15464 leader_election.cc:334] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a936256175c245da8101aab12ceb2c23 (127.7.171.62:36799): Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
I20260809 04:45:33.478729 16318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } }
I20260809 04:45:33.479892 16329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6570f8e34ec04ce8a4eda09c02115105. Latest consensus state: current_term: 2 leader_uuid: "6570f8e34ec04ce8a4eda09c02115105" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:33.480003 16329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:33.480367 16331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:33.482594 16331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:33.482934 16331 catalog_manager.cc:1259] Loaded cluster ID: 98ab283728104e609f7f6e43a857e25c
I20260809 04:45:33.483007 16331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:33.483603 16331 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:33.483999 16331 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: Loaded TSK: 0
I20260809 04:45:33.484774 16331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:45:33.526292 15477 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:33.529853 15477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:33.532938 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.532648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38584)
0809 04:45:33.532767 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.532855 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 04:45:33.533046 15719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [term 2 FOLLOWER]: Refusing update from remote peer 6570f8e34ec04ce8a4eda09c02115105: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 04:45:33.533587 16318 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 04:45:33.533699 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.533479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.0.0.1:38594)
0809 04:45:33.533590 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.533646 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 04:45:33.533857 15464 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 -> Peer a936256175c245da8101aab12ceb2c23 (127.7.171.62:36799): Couldn't send request to peer a936256175c245da8101aab12ceb2c23. Status: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 04:45:33.535074 16321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6570f8e34ec04ce8a4eda09c02115105. Latest consensus state: current_term: 2 leader_uuid: "6570f8e34ec04ce8a4eda09c02115105" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:33.535255 16321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:45:33.536379 16318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6570f8e34ec04ce8a4eda09c02115105" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:33.536530 16318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:33.537055 16321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6570f8e34ec04ce8a4eda09c02115105" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a936256175c245da8101aab12ceb2c23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 36799 } } peers { permanent_uuid: "6570f8e34ec04ce8a4eda09c02115105" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 33893 } attrs { promote: false } } peers { permanent_uuid: "60952733ca774706ad7ab46575a36282" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 43709 } attrs { promote: false } } }
I20260809 04:45:33.537187 16321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60952733ca774706ad7ab46575a36282 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:45:33.537307 16333 mvcc.cc:205] Tried to move back new op lower bound from 7316483004546109440 to 7316483004331278336. Current Snapshot: MvccSnapshot[applied={T|T < 7316483004546109440}]
I20260809 04:45:33.546495 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.542362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36007 (local address 127.0.0.1:55880)
0809 04:45:33.542480 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.542499 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:33.542581 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:33.543097 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:33.543099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:33.543115 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:33.543222 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.543227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.544412 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.544415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.545161 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.545169 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.546048 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.546051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.546143 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.546175 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:45:33.546248 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 04:45:33.546367 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 04:45:33.547062 16337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.542586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55880 (local address 127.7.171.1:36007)
0809 04:45:33.542856 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:33.542861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:33.542875 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:33.542925 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:33.542928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:33.542974 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:33.543361 (+ 387us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:33.543475 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.544306 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.545296 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.545927 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.546045 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.546121 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:33.546873 (+ 752us) server_negotiation.cc:299] Negotiation successful
0809 04:45:33.546963 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":122,"threads_started":1}
I20260809 04:45:33.547587 15831 tablet_service.cc:1467] Processing CreateTablet for tablet 3695d19f7c474107971d5f6cf3273975 (DEFAULT_TABLE table=test-table [id=9468b6652e6b4e428b45ae782e91a180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:33.547895 15831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3695d19f7c474107971d5f6cf3273975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:33.550043 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.542626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:37135 (local address 127.0.0.1:49092)
0809 04:45:33.542776 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.542790 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:33.542880 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:33.545850 (+ 2970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:33.545854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:33.545873 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:33.545958 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.545967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.547630 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.547633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.548654 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.548664 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.549607 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.549611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.549708 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.549732 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:45:33.549815 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 04:45:33.549908 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":64}
I20260809 04:45:33.550294 16338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.542944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:41037 (local address 127.0.0.1:34468)
0809 04:45:33.543680 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.543693 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:33.543775 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:33.545686 (+ 1911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:33.545689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:33.545709 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:33.545849 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.545856 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.547680 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.547683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.548921 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.548932 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.549897 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.549901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.549987 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.550012 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:45:33.550060 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 04:45:33.550123 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":90,"thread_start_us":112,"threads_started":1}
I20260809 04:45:33.550616 16341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.545389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49092 (local address 127.7.171.2:37135)
0809 04:45:33.545644 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:33.545649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:33.545663 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:33.545707 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:33.545710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:33.545751 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:33.545842 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:33.546926 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.547512 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.548786 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.549222 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.549347 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.549414 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:33.550293 (+ 879us) server_negotiation.cc:299] Negotiation successful
0809 04:45:33.550458 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
I20260809 04:45:33.550868 16342 tablet_bootstrap.cc:492] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253: Bootstrap starting.
I20260809 04:45:33.550894 16339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.543322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34468 (local address 127.7.171.3:41037)
0809 04:45:33.545445 (+ 2123us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:33.545450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:33.545464 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:33.545513 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:33.545516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:33.545583 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:33.545670 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:33.546107 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.546784 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.549078 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.549732 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.550550 (+ 818us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.550637 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:33.550677 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 04:45:33.550764 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2042,"thread_start_us":84,"threads_started":1}
I20260809 04:45:33.551417 16150 tablet_service.cc:1467] Processing CreateTablet for tablet 3695d19f7c474107971d5f6cf3273975 (DEFAULT_TABLE table=test-table [id=9468b6652e6b4e428b45ae782e91a180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:33.551503 15982 tablet_service.cc:1467] Processing CreateTablet for tablet 3695d19f7c474107971d5f6cf3273975 (DEFAULT_TABLE table=test-table [id=9468b6652e6b4e428b45ae782e91a180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:33.551702 16150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3695d19f7c474107971d5f6cf3273975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:33.551738 15982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3695d19f7c474107971d5f6cf3273975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:33.551899 16342 tablet_bootstrap.cc:654] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:33.553153 16342 tablet_bootstrap.cc:492] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253: No bootstrap required, opened a new log
I20260809 04:45:33.553242 16342 ts_tablet_manager.cc:1397] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:33.553695 16342 raft_consensus.cc:348] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } } peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } }
I20260809 04:45:33.553815 16342 raft_consensus.cc:374] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:33.553850 16342 raft_consensus.cc:729] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e91f0d374ee4533b1ee5b6f0f016253, State: Initialized, Role: FOLLOWER
I20260809 04:45:33.553990 16342 consensus_queue.cc:260] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } } peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } }
I20260809 04:45:33.554246 16345 tablet_bootstrap.cc:492] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb: Bootstrap starting.
I20260809 04:45:33.554337 16342 ts_tablet_manager.cc:1428] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 04:45:33.555239 16345 tablet_bootstrap.cc:654] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:33.555866 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.554881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.1:50641)
0809 04:45:33.555703 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.555763 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":103,"threads_started":1}
I20260809 04:45:33.556439 16347 tablet_bootstrap.cc:492] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045: Bootstrap starting.
I20260809 04:45:33.556679 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.556442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.1:54203)
0809 04:45:33.556550 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.556617 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 04:45:33.557160 15911 heartbeater.cc:503] Master 127.7.171.61:33893 was elected leader, sending a full tablet report...
I20260809 04:45:33.557603 16347 tablet_bootstrap.cc:654] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:33.558364 16345 tablet_bootstrap.cc:492] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb: No bootstrap required, opened a new log
I20260809 04:45:33.558459 16345 ts_tablet_manager.cc:1397] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 04:45:33.558997 16345 raft_consensus.cc:348] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } } peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } }
I20260809 04:45:33.559113 16345 raft_consensus.cc:374] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:33.559146 16345 raft_consensus.cc:729] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc661cd26ea43069548bd55457a4beb, State: Initialized, Role: FOLLOWER
I20260809 04:45:33.559279 16347 tablet_bootstrap.cc:492] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045: No bootstrap required, opened a new log
I20260809 04:45:33.559274 16345 consensus_queue.cc:260] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } } peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } }
I20260809 04:45:33.559368 16347 ts_tablet_manager.cc:1397] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:33.559626 16345 ts_tablet_manager.cc:1428] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:33.559814 16347 raft_consensus.cc:348] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } } peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } }
I20260809 04:45:33.559918 16347 raft_consensus.cc:374] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:33.559947 16347 raft_consensus.cc:729] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02b1647f9484f87a92108d3618ff045, State: Initialized, Role: FOLLOWER
I20260809 04:45:33.560091 16347 consensus_queue.cc:260] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } } peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } }
I20260809 04:45:33.560436 16347 ts_tablet_manager.cc:1428] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:33.560559 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.560095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.3:32791)
0809 04:45:33.560404 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.560458 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":102,"threads_started":1}
I20260809 04:45:33.561684 16232 heartbeater.cc:503] Master 127.7.171.61:33893 was elected leader, sending a full tablet report...
I20260809 04:45:33.561969 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.560994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.3:46191)
0809 04:45:33.561851 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.561900 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":94,"threads_started":1}
I20260809 04:45:33.563609 16356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.560702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.2:50247)
0809 04:45:33.563449 (+ 2747us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.563509 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2677,"thread_start_us":99,"threads_started":1}
I20260809 04:45:33.564486 16356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.564161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.2:57479)
0809 04:45:33.564354 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.564421 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260809 04:45:33.566407 15912 tablet.cc:2369] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:45:33.696168 16352 raft_consensus.cc:482] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:33.696346 16352 raft_consensus.cc:504] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } } peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } }
I20260809 04:45:33.697588 16352 leader_election.cc:288] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e91f0d374ee4533b1ee5b6f0f016253 (127.7.171.1:36007), d02b1647f9484f87a92108d3618ff045 (127.7.171.2:37135)
I20260809 04:45:33.702010 16341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.698050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:54189 (local address 127.7.171.2:37135)
0809 04:45:33.698229 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:33.698236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:33.698511 (+ 275us) server_negotiation.cc:407] Connection header received
0809 04:45:33.698710 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:33.698714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:33.698773 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:33.698867 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:33.699153 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.699711 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.700927 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.701373 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.701515 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.701587 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:33.701794 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 04:45:33.701873 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:45:33.702010 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.697889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:37135 (local address 127.7.171.3:54189)
0809 04:45:33.698400 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.698412 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:33.698559 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:33.698876 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:33.698880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:33.698897 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:33.699028 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.699033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.699835 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.699841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.700791 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.700802 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.701520 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.701524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.701654 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.701676 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:45:33.701775 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 04:45:33.701854 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":368}
I20260809 04:45:33.702216 16357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.697889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36007 (local address 127.7.171.3:48907)
0809 04:45:33.698092 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.698115 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:33.698281 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:33.698656 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:33.698659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:33.698681 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:33.698840 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.698848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.699702 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.699707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.700799 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:33.700810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.701600 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.701605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:33.701727 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.701749 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:45:33.701857 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 04:45:33.701944 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":24}
I20260809 04:45:33.702348 16337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.698170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:48907 (local address 127.7.171.1:36007)
0809 04:45:33.698349 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:33.698357 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:33.698377 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:45:33.698455 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:33.698457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:33.698506 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:33.698616 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:33.698981 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.699564 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.700983 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.701428 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.701591 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.701667 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:33.701892 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 04:45:33.701975 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:45:33.702608 16014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3695d19f7c474107971d5f6cf3273975" candidate_uuid: "5dc661cd26ea43069548bd55457a4beb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02b1647f9484f87a92108d3618ff045" is_pre_election: true
I20260809 04:45:33.702771 15854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3695d19f7c474107971d5f6cf3273975" candidate_uuid: "5dc661cd26ea43069548bd55457a4beb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" is_pre_election: true
I20260809 04:45:33.702913 15854 raft_consensus.cc:2454] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dc661cd26ea43069548bd55457a4beb in term 0.
I20260809 04:45:33.702926 16014 raft_consensus.cc:2454] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dc661cd26ea43069548bd55457a4beb in term 0.
I20260809 04:45:33.703442 16098 leader_election.cc:302] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5dc661cd26ea43069548bd55457a4beb, 9e91f0d374ee4533b1ee5b6f0f016253, d02b1647f9484f87a92108d3618ff045; no voters:
I20260809 04:45:33.703765 16352 raft_consensus.cc:2781] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:33.703827 16352 raft_consensus.cc:482] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:33.703858 16352 raft_consensus.cc:3037] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:33.705050 16352 raft_consensus.cc:504] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } } peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } }
I20260809 04:45:33.705474 16352 leader_election.cc:288] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [CANDIDATE]: Term 1 election: Requested vote from peers 9e91f0d374ee4533b1ee5b6f0f016253 (127.7.171.1:36007), d02b1647f9484f87a92108d3618ff045 (127.7.171.2:37135)
I20260809 04:45:33.705855 16014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3695d19f7c474107971d5f6cf3273975" candidate_uuid: "5dc661cd26ea43069548bd55457a4beb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d02b1647f9484f87a92108d3618ff045"
I20260809 04:45:33.705899 15854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3695d19f7c474107971d5f6cf3273975" candidate_uuid: "5dc661cd26ea43069548bd55457a4beb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e91f0d374ee4533b1ee5b6f0f016253"
I20260809 04:45:33.706000 16014 raft_consensus.cc:3037] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:33.706039 15854 raft_consensus.cc:3037] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:33.707111 15854 raft_consensus.cc:2454] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dc661cd26ea43069548bd55457a4beb in term 1.
I20260809 04:45:33.707118 16014 raft_consensus.cc:2454] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dc661cd26ea43069548bd55457a4beb in term 1.
I20260809 04:45:33.707405 16098 leader_election.cc:302] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dc661cd26ea43069548bd55457a4beb, 9e91f0d374ee4533b1ee5b6f0f016253; no voters:
I20260809 04:45:33.707621 16352 raft_consensus.cc:2781] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:33.707859 16352 raft_consensus.cc:686] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 1 LEADER]: Becoming Leader. State: Replica: 5dc661cd26ea43069548bd55457a4beb, State: Running, Role: LEADER
I20260809 04:45:33.708411 16352 consensus_queue.cc:237] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } } peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } }
I20260809 04:45:33.708654 16357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.708365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.3:54523)
0809 04:45:33.708492 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.708554 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 04:45:33.711021 15480 catalog_manager.cc:5495] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb reported cstate change: term changed from 0 to 1, leader changed from <none> to 5dc661cd26ea43069548bd55457a4beb (127.7.171.3). New cstate: current_term: 1 leader_uuid: "5dc661cd26ea43069548bd55457a4beb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dc661cd26ea43069548bd55457a4beb" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41037 } health_report { overall_health: HEALTHY } } }
I20260809 04:45:33.740177 16339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.736043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34482 (local address 127.7.171.3:41037)
0809 04:45:33.736287 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:33.736292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:33.736309 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:45:33.736546 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:33.736549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:33.736618 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:33.736750 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:33.737017 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.737586 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.738544 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:33.738805 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:33.738940 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:33.739241 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:33.739715 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:33.739855 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:33.739933 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 04:45:33.739996 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260809 04:45:33.743283 15854 raft_consensus.cc:1261] T 3695d19f7c474107971d5f6cf3273975 P 9e91f0d374ee4533b1ee5b6f0f016253 [term 1 FOLLOWER]: Refusing update from remote peer 5dc661cd26ea43069548bd55457a4beb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:33.743520 16014 raft_consensus.cc:1261] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045 [term 1 FOLLOWER]: Refusing update from remote peer 5dc661cd26ea43069548bd55457a4beb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:33.743887 16352 consensus_queue.cc:1035] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e91f0d374ee4533b1ee5b6f0f016253" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:33.744168 16358 consensus_queue.cc:1035] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d02b1647f9484f87a92108d3618ff045" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:33.747180 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.746868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.1:55249)
0809 04:45:33.747010 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.747084 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 04:45:33.747329 16072 heartbeater.cc:503] Master 127.7.171.61:33893 was elected leader, sending a full tablet report...
I20260809 04:45:33.748112 16356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.747793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.2:41205)
0809 04:45:33.747937 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.748005 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 04:45:33.749503 16357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.749214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.3:45747)
0809 04:45:33.749351 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.749414 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 04:45:33.751289 16357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.749992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.3:51573)
0809 04:45:33.751167 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.751219 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1101}
I20260809 04:45:33.752329 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.750942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.1:44139)
0809 04:45:33.752203 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.752257 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1197}
I20260809 04:45:33.753410 16356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.753157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:36799 (local address 127.7.171.2:55341)
0809 04:45:33.753275 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.753335 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 04:45:33.761946 15477 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:33.762224 15477 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:33.766494 15477 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6570f8e34ec04ce8a4eda09c02115105: Sending DeleteTablet for 3 replicas of tablet 3695d19f7c474107971d5f6cf3273975
I20260809 04:45:33.767289 15982 tablet_service.cc:1514] Processing DeleteTablet for tablet 3695d19f7c474107971d5f6cf3273975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:33 UTC) from {username='oryx', principal='oryx/127.7.171.61@KRBTEST.COM'} at 127.0.0.1:49092
I20260809 04:45:33.767525 15831 tablet_service.cc:1514] Processing DeleteTablet for tablet 3695d19f7c474107971d5f6cf3273975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:33 UTC) from {username='oryx', principal='oryx/127.7.171.61@KRBTEST.COM'} at 127.0.0.1:55880
I20260809 04:45:33.767560 16150 tablet_service.cc:1514] Processing DeleteTablet for tablet 3695d19f7c474107971d5f6cf3273975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:33 UTC) from {username='oryx', principal='oryx/127.7.171.61@KRBTEST.COM'} at 127.0.0.1:34468
I20260809 04:45:33.767673 16375 tablet_replica.cc:330] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045: stopping tablet replica
I20260809 04:45:33.767815 16375 raft_consensus.cc:2229] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:33.767956 16375 raft_consensus.cc:2258] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:33.768573 16378 tablet_replica.cc:330] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb: stopping tablet replica
I20260809 04:45:33.768697 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 15656
I20260809 04:45:33.769451 16378 raft_consensus.cc:2229] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:33.769876 16378 raft_consensus.cc:2258] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:33.769194 16375 ts_tablet_manager.cc:1905] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:33.774291 16375 ts_tablet_manager.cc:1918] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:33.774406 16375 log.cc:1196] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/3695d19f7c474107971d5f6cf3273975
I20260809 04:45:33.775346 16378 ts_tablet_manager.cc:1905] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:33.779386 16378 ts_tablet_manager.cc:1918] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:33.779493 16378 log.cc:1196] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/3695d19f7c474107971d5f6cf3273975
I20260809 04:45:33.779830 16378 ts_tablet_manager.cc:1939] T 3695d19f7c474107971d5f6cf3273975 P 5dc661cd26ea43069548bd55457a4beb: Deleting consensus metadata
I20260809 04:45:33.780386 16375 ts_tablet_manager.cc:1939] T 3695d19f7c474107971d5f6cf3273975 P d02b1647f9484f87a92108d3618ff045: Deleting consensus metadata
I20260809 04:45:33.780808 15462 catalog_manager.cc:4841] TS 5dc661cd26ea43069548bd55457a4beb (127.7.171.3:41037): tablet 3695d19f7c474107971d5f6cf3273975 (table test-table [id=9468b6652e6b4e428b45ae782e91a180]) successfully deleted
I20260809 04:45:33.781303 15462 catalog_manager.cc:4841] TS d02b1647f9484f87a92108d3618ff045 (127.7.171.2:37135): tablet 3695d19f7c474107971d5f6cf3273975 (table test-table [id=9468b6652e6b4e428b45ae782e91a180]) successfully deleted
I20260809 04:45:33.784487 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 15918
I20260809 04:45:33.784693 16338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:33.784318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36007 (local address 127.0.0.1:55884)
0809 04:45:33.784513 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:33.784589 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:36007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260809 04:45:33.790701 15461 catalog_manager.cc:4584] TS 9e91f0d374ee4533b1ee5b6f0f016253 (127.7.171.1:36007): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3695d19f7c474107971d5f6cf3273975: Network error: Client connection negotiation failed: client connection to 127.7.171.1:36007: connect: Connection refused (error 111)
I20260809 04:45:33.794773 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 16077
I20260809 04:45:33.804373 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 15322
I20260809 04:45:33.813455 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 15448
2026-08-09T04:45:33Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5677 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:33 dist-test-slave-1zqn krb5kdc[16382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:33 dist-test-slave-1zqn krb5kdc[16382](info): set up 2 sockets
Aug 09 04:45:33 dist-test-slave-1zqn krb5kdc[16382](info): commencing operation
krb5kdc: starting...
W20260809 04:45:34.938257 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:35 dist-test-slave-1zqn krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.62.keytab.
Entry for principal oryx/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.62.keytab.
I20260809 04:45:35.076349 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:39447
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.62.keytab
--principal=oryx/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:38033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:39447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:35.189056 16397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:35.189383 16397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:35.189462 16397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:35.193450 16397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:35.193545 16397 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:45:35.193578 16397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:35.193619 16397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:35.193656 16397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:35.193694 16397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:35.199052 16397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:39447
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.62.keytab
--principal=oryx/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:39447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:35.200214 16397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:35.201280 16397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:35.210510 16405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:35.210574 16406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:35.210541 16409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:35.210804 16397 server_base.cc:1034] running on GCE node
I20260809 04:45:35.211457 16397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:35.212754 16397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:35.213965 16397 hybrid_clock.cc:648] HybridClock initialized: now 1786250735213940 us; error 35 us; skew 500 ppm
Aug 09 04:45:35 dist-test-slave-1zqn krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250735, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:35.217389 16397 init.cc:377] Logged in from keytab as oryx/127.7.171.62@KRBTEST.COM (short username oryx)
I20260809 04:45:35.218449 16397 webserver.cc:460] Webserver started at http://127.7.171.62:34563/ using document root <none> and password file <none>
I20260809 04:45:35.219102 16397 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:35.219208 16397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:35.219471 16397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:35.221794 16397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "cd8a8f31194d491cb4202bcae9e1ac49"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "a723e1f19eb6d48977dc5b2b721661e7"
server_key_iv: "efa99d1fd564d99d44a2630c2683b518"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.222425 16397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "cd8a8f31194d491cb4202bcae9e1ac49"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "a723e1f19eb6d48977dc5b2b721661e7"
server_key_iv: "efa99d1fd564d99d44a2630c2683b518"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.226646 16397 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 04:45:35.229349 16418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:35.230693 16397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:45:35.230854 16397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "cd8a8f31194d491cb4202bcae9e1ac49"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "a723e1f19eb6d48977dc5b2b721661e7"
server_key_iv: "efa99d1fd564d99d44a2630c2683b518"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.230984 16397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:35.253391 16397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:35.256532 16397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:35.256788 16397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:35.266027 16397 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:39447
I20260809 04:45:35.266077 16495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:39447 every 8 connection(s)
I20260809 04:45:35.267114 16397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:35.270030 16496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:35.274168 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 16397
I20260809 04:45:35.274283 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:35.274621 7852 external_mini_cluster.cc:1428] Setting key 8d09cbdbb49cfea35df67101583c4bcd
I20260809 04:45:35.276275 16496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49: Bootstrap starting.
I20260809 04:45:35.279182 16496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:35.280182 16496 log.cc:824] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:35.282356 16496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49: No bootstrap required, opened a new log
Aug 09 04:45:35 dist-test-slave-1zqn krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:35.286567 16496 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8a8f31194d491cb4202bcae9e1ac49" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39447 } }
I20260809 04:45:35.286883 16496 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:35.286957 16496 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd8a8f31194d491cb4202bcae9e1ac49, State: Initialized, Role: FOLLOWER
I20260809 04:45:35.287470 16496 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8a8f31194d491cb4202bcae9e1ac49" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39447 } }
I20260809 04:45:35.287650 16496 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:35.287757 16496 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:35.287861 16496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:35.289089 16496 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8a8f31194d491cb4202bcae9e1ac49" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39447 } }
I20260809 04:45:35.289494 16496 leader_election.cc:302] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd8a8f31194d491cb4202bcae9e1ac49; no voters:
I20260809 04:45:35.289899 16496 leader_election.cc:288] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:35.290022 16502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:35.290328 16502 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [term 1 LEADER]: Becoming Leader. State: Replica: cd8a8f31194d491cb4202bcae9e1ac49, State: Running, Role: LEADER
I20260809 04:45:35.290644 16502 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8a8f31194d491cb4202bcae9e1ac49" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39447 } }
I20260809 04:45:35.291061 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:35.276250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54582 (local address 127.7.171.62:39447)
0809 04:45:35.276764 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:35.276784 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:35.276827 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:45:35.277750 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:35.277774 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:35.278212 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:35.278595 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:35.279598 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.281025 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.281967 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.282672 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.282865 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:35.285463 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:35.285469 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:35.285486 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:35.285518 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:35.287539 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:35.288084 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:35.288093 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:35.288102 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:35.288250 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:35.288530 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:35.288533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:35.288535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:35.289083 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:35.289283 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:35.289596 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 04:45:35.290109 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":196,"threads_started":1}
I20260809 04:45:35.291208 16496 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:35.291921 16508 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd8a8f31194d491cb4202bcae9e1ac49. Latest consensus state: current_term: 1 leader_uuid: "cd8a8f31194d491cb4202bcae9e1ac49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8a8f31194d491cb4202bcae9e1ac49" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39447 } } }
I20260809 04:45:35.292089 16508 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:35.292397 16503 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd8a8f31194d491cb4202bcae9e1ac49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8a8f31194d491cb4202bcae9e1ac49" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39447 } } }
I20260809 04:45:35.292503 16503 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49 [sys.catalog]: This master's current role is: LEADER
W20260809 04:45:35.296334 16521 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:45:35.296424 16521 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:45:35.296492 16514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:35.297411 16514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:35.303371 16514 catalog_manager.cc:1347] Generated new cluster ID: 44b3fd26118c48c480dff349561f167c
I20260809 04:45:35.303442 16514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:35.320952 16514 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:35.322358 16514 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:35.329038 16514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49: Generated new TSK 0
I20260809 04:45:35.329864 16514 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab.
Entry for principal oryx/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab.
I20260809 04:45:35.363329 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:39447
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab
--principal=oryx/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:38033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:35.477468 16529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:35.477738 16529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:35.477783 16529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:35.481848 16529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:35.481910 16529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:35.481949 16529 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:45:35.481983 16529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:35.487207 16529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab
--principal=oryx/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.171.62:39447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:35.488334 16529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:35.489398 16529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:35.499418 16537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:35.499429 16535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:35.499704 16529 server_base.cc:1034] running on GCE node
W20260809 04:45:35.499429 16534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:35.500260 16529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:35.500941 16529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:35.502135 16529 hybrid_clock.cc:648] HybridClock initialized: now 1786250735502106 us; error 51 us; skew 500 ppm
Aug 09 04:45:35 dist-test-slave-1zqn krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250735, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:35.505043 16529 init.cc:377] Logged in from keytab as oryx/127.7.171.1@KRBTEST.COM (short username oryx)
I20260809 04:45:35.505944 16529 webserver.cc:460] Webserver started at http://127.7.171.1:34835/ using document root <none> and password file <none>
I20260809 04:45:35.506529 16529 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:35.506628 16529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:35.506875 16529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:35.509085 16529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "0b977419893e4504a59f0561b8a8efa8"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "55ae95544f596e766c3a705860c3d6d1"
server_key_iv: "69bc62546566e21d85a070a3a45e960f"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.509670 16529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "0b977419893e4504a59f0561b8a8efa8"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "55ae95544f596e766c3a705860c3d6d1"
server_key_iv: "69bc62546566e21d85a070a3a45e960f"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.513667 16529 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:35.516253 16545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:35.517478 16529 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:45:35.517642 16529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "0b977419893e4504a59f0561b8a8efa8"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "55ae95544f596e766c3a705860c3d6d1"
server_key_iv: "69bc62546566e21d85a070a3a45e960f"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.517760 16529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:35.538388 16529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:35.541618 16529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:35.541838 16529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:35.543483 16529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:35.543560 16529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:35.543668 16529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:35.543718 16529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:45:35 dist-test-slave-1zqn krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250735, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:35.555230 16529 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:45511
I20260809 04:45:35.555275 16681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:45511 every 8 connection(s)
I20260809 04:45:35.556435 16529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:35.558729 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:35.544719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:57307 (local address 127.7.171.62:39447)
0809 04:45:35.544892 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:35.544899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:35.545925 (+ 1026us) server_negotiation.cc:407] Connection header received
0809 04:45:35.547329 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:35.547334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:35.547413 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:35.547545 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:35.549009 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.549661 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.551094 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.551346 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.551500 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:35.554486 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:35.554490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:35.554494 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:35.554530 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:35.556405 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:35.557077 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:35.557081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:35.557083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:35.557187 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:35.557579 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:35.557583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:35.557584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:35.557784 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:35.557898 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:35.558396 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 04:45:35.558527 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:45:35.559476 16555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:35.545174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:39447 (local address 127.7.171.1:57307)
0809 04:45:35.545723 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:35.545786 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:35.547056 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:35.547742 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:35.547750 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:35.548279 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:35.548827 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:35.548843 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.549819 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.549823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:35.550948 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:35.550958 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.551575 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.551580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:35.551772 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:35.552467 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:35.552485 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:35.554289 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:35.556571 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:35.556577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:35.556598 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:35.556944 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:35.557317 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:35.557321 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:35.557323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:35.557465 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:35.557899 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:35.557906 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:35.558161 (+ 255us) client_negotiation.cc:769] Sending connection context
0809 04:45:35.558352 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 04:45:35.558680 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":125,"threads_started":1}
I20260809 04:45:35.560940 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 16529
I20260809 04:45:35.561007 16682 heartbeater.cc:347] Connected to a master server at 127.7.171.62:39447
I20260809 04:45:35.561081 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:35.561169 16682 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:35.561421 7852 external_mini_cluster.cc:1428] Setting key 7f84bf7e6573445c46105a724ae9fcfb
I20260809 04:45:35.561738 16682 heartbeater.cc:511] Master 127.7.171.62:39447 requested a full tablet report, sending...
I20260809 04:45:35.563414 16439 ts_manager.cc:194] Registered new tserver with Master: 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511)
I20260809 04:45:35.565052 16439 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:57307
I20260809 04:45:35.572253 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:35.563156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:39447 (local address 127.7.171.1:44889)
0809 04:45:35.564403 (+ 1247us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:35.564415 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:35.564527 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:35.564790 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:35.564794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:35.564993 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:35.565102 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:35.565108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.565932 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.565936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:35.566803 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:35.566815 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.567494 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.567497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:35.567592 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:35.568189 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:35.568209 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:35.568701 (+ 492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:35.570729 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:35.570731 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:35.570733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:35.570928 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:35.571234 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:35.571236 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:35.571237 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:35.571314 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:35.571752 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:35.571754 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:35.571828 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 04:45:35.571908 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:45:35.572017 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1160,"thread_start_us":87,"threads_started":1}
I20260809 04:45:35.572402 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:35.564191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:44889 (local address 127.7.171.62:39447)
0809 04:45:35.564320 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:35.564324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:35.564486 (+ 162us) server_negotiation.cc:407] Connection header received
0809 04:45:35.564653 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:35.564655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:35.564691 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:35.564773 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:35.565301 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.565797 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.566953 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.567349 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.567475 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:35.568870 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:35.568876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:35.568881 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:35.568910 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:35.570612 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:35.571029 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:35.571033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:35.571035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:35.571135 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:35.571403 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:35.571406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:35.571408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:35.571651 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:35.571763 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:35.572008 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 04:45:35.572216 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for oryx/127.7.171.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab.
Entry for principal oryx/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab.
I20260809 04:45:35.595239 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:39447
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab
--principal=oryx/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:38033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:35.712674 16691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:35.712937 16691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:35.712983 16691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:35.716854 16691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:35.716908 16691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:35.716952 16691 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:45:35.716985 16691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:35.722395 16691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab
--principal=oryx/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.171.62:39447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:35.723369 16691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:35.724534 16691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:35.734239 16700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:35.734249 16696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:35.734247 16698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:35.734668 16691 server_base.cc:1034] running on GCE node
I20260809 04:45:35.735170 16691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:35.735848 16691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:35.737064 16691 hybrid_clock.cc:648] HybridClock initialized: now 1786250735737024 us; error 59 us; skew 500 ppm
Aug 09 04:45:35 dist-test-slave-1zqn krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250735, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:35.740152 16691 init.cc:377] Logged in from keytab as oryx/127.7.171.2@KRBTEST.COM (short username oryx)
I20260809 04:45:35.741025 16691 webserver.cc:460] Webserver started at http://127.7.171.2:37833/ using document root <none> and password file <none>
I20260809 04:45:35.741593 16691 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:35.741688 16691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:35.741922 16691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:35.744175 16691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "430f845c3f814252b67dcab7c78b0332"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "6d160f053398a2e39118fb76fba4d366"
server_key_iv: "f8c97707b2245ca11fcb1b4dda76c397"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.744756 16691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "430f845c3f814252b67dcab7c78b0332"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "6d160f053398a2e39118fb76fba4d366"
server_key_iv: "f8c97707b2245ca11fcb1b4dda76c397"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.748571 16691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:35.751085 16709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:35.752358 16691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:35.752489 16691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "430f845c3f814252b67dcab7c78b0332"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "6d160f053398a2e39118fb76fba4d366"
server_key_iv: "f8c97707b2245ca11fcb1b4dda76c397"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.752611 16691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:35.772023 16691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:35.775674 16691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:35.775903 16691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:35.777448 16691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:35.777531 16691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:35.777601 16691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:35.777652 16691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:45:35 dist-test-slave-1zqn krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250735, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:35.789395 16691 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:36701
I20260809 04:45:35.789497 16844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:36701 every 8 connection(s)
I20260809 04:45:35.790586 16691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:35.793354 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:35.778812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:44701 (local address 127.7.171.62:39447)
0809 04:45:35.778955 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:35.778960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:35.780150 (+ 1190us) server_negotiation.cc:407] Connection header received
0809 04:45:35.781510 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:35.781517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:35.781594 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:35.781703 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:35.783142 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.783781 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.785610 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.785856 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.786008 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:35.789257 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:35.789261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:35.789263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:35.789290 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:35.791022 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:35.791643 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:35.791649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:35.791651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:35.791742 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:35.792191 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:35.792194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:35.792196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:35.792454 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:35.792563 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:35.793044 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 04:45:35.793166 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:45:35.793543 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 16691
I20260809 04:45:35.793648 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:35.793972 7852 external_mini_cluster.cc:1428] Setting key 473c252f19b288c9bb32d15cd18ef94c
I20260809 04:45:35.794263 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:35.779371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:39447 (local address 127.7.171.2:44701)
0809 04:45:35.779914 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:35.779971 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:35.781196 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:35.781931 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:35.781942 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:35.782369 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:35.782978 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:35.782993 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.783921 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.783925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:35.785492 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:35.785505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.786073 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.786077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:35.786304 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:35.787035 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:35.787091 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:35.789083 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:35.791147 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:35.791154 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:35.791168 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:35.791509 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:35.791851 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:35.791854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:35.791856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:35.792097 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:35.792562 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:35.792569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:35.792809 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 04:45:35.792987 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 04:45:35.793303 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":131,"threads_started":1}
I20260809 04:45:35.795488 16845 heartbeater.cc:347] Connected to a master server at 127.7.171.62:39447
I20260809 04:45:35.795670 16845 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:35.796285 16845 heartbeater.cc:511] Master 127.7.171.62:39447 requested a full tablet report, sending...
I20260809 04:45:35.797396 16439 ts_manager.cc:194] Registered new tserver with Master: 430f845c3f814252b67dcab7c78b0332 (127.7.171.2:36701)
I20260809 04:45:35.798126 16439 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:44701
WARNING: no policy specified for oryx/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:45:35.806483 16852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:35.798099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:39447 (local address 127.7.171.2:43117)
0809 04:45:35.798430 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:35.798452 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:35.798562 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:35.798807 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:35.798810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:35.799017 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:35.799160 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:35.799166 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.800251 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.800258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:35.801287 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:35.801302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.801749 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.801754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:35.801871 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:35.802366 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:35.802386 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:35.802872 (+ 486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:35.804680 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:35.804687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:35.804691 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:35.804968 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:35.805267 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:35.805272 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:35.805277 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:35.805380 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:35.806017 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:35.806022 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:35.806128 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 04:45:35.806237 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 04:45:35.806342 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260809 04:45:35.806509 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:35.798130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:43117 (local address 127.7.171.62:39447)
0809 04:45:35.798263 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:35.798269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:35.798523 (+ 254us) server_negotiation.cc:407] Connection header received
0809 04:45:35.798660 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:35.798663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:35.798703 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:35.798792 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:35.799274 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.800101 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.801406 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:35.801623 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:35.801708 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:35.803070 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:35.803073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:35.803076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:35.803096 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:35.804571 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:35.805063 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:35.805065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:35.805067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:35.805157 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:35.805601 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:35.805604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:35.805605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:35.805907 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:35.805994 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:35.806302 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 04:45:35.806398 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab.
Entry for principal oryx/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab.
I20260809 04:45:35.828642 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:39447
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab
--principal=oryx/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:38033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:35.947186 16854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:35.947484 16854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:35.947579 16854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:35.951713 16854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:35.951809 16854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:35.951925 16854 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:45:35.952013 16854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:35.957369 16854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab
--principal=oryx/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.171.62:39447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:35.958387 16854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:35.959429 16854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:35.969419 16865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:35.969488 16860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:35.969488 16861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:35.969548 16854 server_base.cc:1034] running on GCE node
I20260809 04:45:35.970082 16854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:35.970713 16854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:35.971912 16854 hybrid_clock.cc:648] HybridClock initialized: now 1786250735971888 us; error 48 us; skew 500 ppm
Aug 09 04:45:35 dist-test-slave-1zqn krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250735, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:35.974759 16854 init.cc:377] Logged in from keytab as oryx/127.7.171.3@KRBTEST.COM (short username oryx)
I20260809 04:45:35.975754 16854 webserver.cc:460] Webserver started at http://127.7.171.3:33787/ using document root <none> and password file <none>
I20260809 04:45:35.976387 16854 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:35.976493 16854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:35.976732 16854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:35.978814 16854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "50e73d3e760147c6a0217469e1229e77"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "8ebd7f031151269af21973ce81fbaa19"
server_key_iv: "82ada2302e91f2714e48cf139ce746da"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.979411 16854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "50e73d3e760147c6a0217469e1229e77"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "8ebd7f031151269af21973ce81fbaa19"
server_key_iv: "82ada2302e91f2714e48cf139ce746da"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.983201 16854 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:45:35.985780 16873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:35.987064 16854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:45:35.987214 16854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "50e73d3e760147c6a0217469e1229e77"
format_stamp: "Formatted at 2026-08-09 04:45:35 on dist-test-slave-1zqn"
server_key: "8ebd7f031151269af21973ce81fbaa19"
server_key_iv: "82ada2302e91f2714e48cf139ce746da"
server_key_version: "encryptionkey@0"
I20260809 04:45:35.987334 16854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:36.010545 16854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:36.014015 16854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:36.014263 16854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:36.015856 16854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:36.015945 16854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:36.016014 16854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:36.016040 16854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:45:36 dist-test-slave-1zqn krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250735, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:36.027781 16854 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:42213
I20260809 04:45:36.027853 17005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:42213 every 8 connection(s)
I20260809 04:45:36.028803 16854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:36.030829 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.016793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:42963 (local address 127.7.171.62:39447)
0809 04:45:36.016924 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.016927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.018059 (+ 1132us) server_negotiation.cc:407] Connection header received
0809 04:45:36.019171 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.019175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.019245 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.019325 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:36.021160 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.021942 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.023312 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.023613 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.023738 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.026964 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:36.026971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:36.026977 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:36.027017 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:36.028731 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:36.029289 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:36.029293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:36.029295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:36.029381 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:36.029790 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:36.029796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:36.029800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:36.030017 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:36.030117 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.030594 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.030700 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:45:36.031610 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.017380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:39447 (local address 127.7.171.3:42963)
0809 04:45:36.017889 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.017934 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.018890 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.019817 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.019825 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.020390 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:36.020984 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.020998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.022135 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.022138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.023169 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.023179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.023797 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.023800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.023979 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.024917 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:36.024990 (+ 73us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:36.026751 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:36.028894 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:36.028900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:36.028915 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:36.029177 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:36.029481 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:36.029486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:36.029488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:36.029667 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:36.030135 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:36.030140 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:36.030357 (+ 217us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.030539 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.030785 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":127,"threads_started":1}
I20260809 04:45:36.033047 17006 heartbeater.cc:347] Connected to a master server at 127.7.171.62:39447
I20260809 04:45:36.033301 17006 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:36.033898 17006 heartbeater.cc:511] Master 127.7.171.62:39447 requested a full tablet report, sending...
I20260809 04:45:36.035064 16438 ts_manager.cc:194] Registered new tserver with Master: 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213)
I20260809 04:45:36.035873 16438 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:42963
I20260809 04:45:36.036577 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 16854
I20260809 04:45:36.036721 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:36.037065 7852 external_mini_cluster.cc:1428] Setting key a49755293b7b0cb0d83359e4abd18033
I20260809 04:45:36.039323 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:45:36.046626 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.037742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:37847 (local address 127.7.171.62:39447)
0809 04:45:36.037871 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.037875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.038264 (+ 389us) server_negotiation.cc:407] Connection header received
0809 04:45:36.038405 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.038408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.038458 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.038552 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:36.039600 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.040234 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.041223 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.041616 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.041749 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.042898 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:36.042906 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:36.042913 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:36.042958 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:36.044809 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:36.045294 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:36.045300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:36.045305 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:36.045407 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:36.045702 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:36.045707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:36.045710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:36.045969 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:36.046096 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.046404 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.046506 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:45:36.046818 17014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.037718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:39447 (local address 127.7.171.3:37847)
0809 04:45:36.038175 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.038189 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.038307 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.039119 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.039121 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.039308 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:36.039465 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.039474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.040377 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.040381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.041085 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.041094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.041753 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.041756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.041831 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.042398 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:36.042413 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:36.042750 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:36.044948 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:36.044952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:36.044955 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:36.045183 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:36.045507 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:36.045509 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:36.045511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:36.045603 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:36.046106 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:36.046112 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:36.046216 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.046340 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.046493 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":93,"threads_started":1}
Aug 09 04:45:36 dist-test-slave-1zqn krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250736, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:45:36 dist-test-slave-1zqn krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:36 dist-test-slave-1zqn krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:45:36.067451 16500 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:36.060363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54598 (local address 127.7.171.62:39447)
0809 04:45:36.060484 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.060489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.060503 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:36.060545 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.060548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.060587 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.060666 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:36.061456 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.062139 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.063341 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.063763 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.063871 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.067323 (+ 3452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54598: BlockingRecv error: recv got EOF from 127.0.0.1:54598 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 04:45:36 dist-test-slave-1zqn krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250736, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:36.078066 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.069368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.7.171.62:39447)
0809 04:45:36.069494 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.069497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.069681 (+ 184us) server_negotiation.cc:407] Connection header received
0809 04:45:36.069779 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.069781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.069820 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.069889 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:36.070560 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.071069 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.071874 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.072145 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.072221 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.074680 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:36.074683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:36.074685 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:36.074709 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:36.076249 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:36.076830 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:36.076841 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:36.076843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:36.077009 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:36.077273 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:36.077275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:36.077276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:36.077495 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:36.077601 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.077838 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.077941 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:45:36.081396 16438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:36.084613 16438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:36.099908 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.093750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:42213 (local address 127.0.0.1:58888)
0809 04:45:36.094395 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.094448 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.094581 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.095722 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.095728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.095752 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:36.095898 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.095904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.097182 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.097187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.098377 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.098387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.099335 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.099339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.099457 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.099498 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.099615 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.099731 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":114,"threads_started":1}
I20260809 04:45:36.099921 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.093871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58888 (local address 127.7.171.3:42213)
0809 04:45:36.095241 (+ 1370us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.095253 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.095276 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:45:36.095351 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.095355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.095549 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.095706 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:36.096033 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.097058 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.098504 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.099211 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.099320 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.099482 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.099664 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.099763 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1255,"thread_start_us":140,"threads_started":1}
I20260809 04:45:36.100544 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.094079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:45511 (local address 127.0.0.1:51028)
0809 04:45:36.095432 (+ 1353us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.095441 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.095505 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.095898 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.095901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.095930 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:36.096043 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.096069 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.097534 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.097537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.098775 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.098785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.100196 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.100199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.100280 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.100300 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.100352 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.100401 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1307,"thread_start_us":86,"threads_started":1}
I20260809 04:45:36.101022 17037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.094572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51028 (local address 127.7.171.1:45511)
0809 04:45:36.094987 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.094997 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.095526 (+ 529us) server_negotiation.cc:407] Connection header received
0809 04:45:36.095605 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.095612 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.095779 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.095918 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:36.096233 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.097403 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.098912 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.099726 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.099832 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.099967 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.100752 (+ 785us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.100856 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":101,"threads_started":1}
I20260809 04:45:36.103119 16932 tablet_service.cc:1467] Processing CreateTablet for tablet 636440feff8b470684f4af60730dc79c (DEFAULT_TABLE table=test-table [id=36c0e892e9d842da83c39f31c08133c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:36.103781 16605 tablet_service.cc:1467] Processing CreateTablet for tablet 636440feff8b470684f4af60730dc79c (DEFAULT_TABLE table=test-table [id=36c0e892e9d842da83c39f31c08133c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:36.104301 16932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 636440feff8b470684f4af60730dc79c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.104725 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 636440feff8b470684f4af60730dc79c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.107301 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.093786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:36701 (local address 127.0.0.1:49828)
0809 04:45:36.095142 (+ 1356us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.095158 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.095267 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.096244 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.096247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.096265 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:36.096404 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.096412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.097809 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.097812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.098929 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.098936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.099936 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.099942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.106971 (+ 7029us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.107004 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.107069 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.107161 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1283,"thread_start_us":67,"threads_started":1}
I20260809 04:45:36.108148 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.094552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49828 (local address 127.7.171.2:36701)
0809 04:45:36.095754 (+ 1202us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.095767 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.095786 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:45:36.095855 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.095860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.096048 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.096260 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:36.096556 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.097689 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.099042 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.099782 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.107630 (+ 7848us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.107820 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.107878 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.107975 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1097,"thread_start_us":128,"threads_started":1}
I20260809 04:45:36.111109 17039 tablet_bootstrap.cc:492] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: Bootstrap starting.
I20260809 04:45:36.110929 16770 tablet_service.cc:1467] Processing CreateTablet for tablet 636440feff8b470684f4af60730dc79c (DEFAULT_TABLE table=test-table [id=36c0e892e9d842da83c39f31c08133c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:36.112221 16770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 636440feff8b470684f4af60730dc79c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.112774 17040 tablet_bootstrap.cc:492] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: Bootstrap starting.
I20260809 04:45:36.113984 17039 tablet_bootstrap.cc:654] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.114728 17039 log.cc:824] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:36.114912 17040 tablet_bootstrap.cc:654] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.115763 17040 log.cc:824] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:36.116847 17039 tablet_bootstrap.cc:492] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: No bootstrap required, opened a new log
I20260809 04:45:36.117061 17039 ts_tablet_manager.cc:1397] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260809 04:45:36.117835 17040 tablet_bootstrap.cc:492] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: No bootstrap required, opened a new log
I20260809 04:45:36.118026 17040 ts_tablet_manager.cc:1397] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:45:36.120688 17044 tablet_bootstrap.cc:492] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: Bootstrap starting.
I20260809 04:45:36.120522 17039 raft_consensus.cc:348] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } }
I20260809 04:45:36.120860 17039 raft_consensus.cc:374] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.120921 17039 raft_consensus.cc:729] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50e73d3e760147c6a0217469e1229e77, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.121400 17039 consensus_queue.cc:260] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } }
I20260809 04:45:36.122143 17006 heartbeater.cc:503] Master 127.7.171.62:39447 was elected leader, sending a full tablet report...
I20260809 04:45:36.122545 17039 ts_tablet_manager.cc:1428] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260809 04:45:36.123261 17044 tablet_bootstrap.cc:654] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.123971 17044 log.cc:824] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:36.124176 17040 raft_consensus.cc:348] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } }
I20260809 04:45:36.124435 17040 raft_consensus.cc:374] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.124511 17040 raft_consensus.cc:729] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b977419893e4504a59f0561b8a8efa8, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.124956 17040 consensus_queue.cc:260] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } }
I20260809 04:45:36.126158 17044 tablet_bootstrap.cc:492] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: No bootstrap required, opened a new log
I20260809 04:45:36.126345 17044 ts_tablet_manager.cc:1397] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:45:36.126513 17040 ts_tablet_manager.cc:1428] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260809 04:45:36.126880 16682 heartbeater.cc:503] Master 127.7.171.62:39447 was elected leader, sending a full tablet report...
I20260809 04:45:36.129990 17044 raft_consensus.cc:348] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } }
I20260809 04:45:36.130183 17044 raft_consensus.cc:374] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.130223 17044 raft_consensus.cc:729] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 430f845c3f814252b67dcab7c78b0332, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.130585 17044 consensus_queue.cc:260] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } }
I20260809 04:45:36.131167 16845 heartbeater.cc:503] Master 127.7.171.62:39447 was elected leader, sending a full tablet report...
I20260809 04:45:36.131373 17044 ts_tablet_manager.cc:1428] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260809 04:45:36.280117 17007 tablet.cc:2369] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:45:36.292502 16846 tablet.cc:2369] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:45:36.308213 16683 tablet.cc:2369] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:45:36.363968 17048 raft_consensus.cc:482] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:36.364259 17048 raft_consensus.cc:504] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } }
I20260809 04:45:36.365438 17048 leader_election.cc:288] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213), 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511)
I20260809 04:45:36.370131 17051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.365789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:42213 (local address 127.7.171.2:50235)
0809 04:45:36.366375 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.366392 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.366506 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.366798 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.366801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.366816 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:36.367045 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.367052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.368021 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.368024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.368985 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.368996 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.369650 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.369654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.369750 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.369822 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.369882 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.369971 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":99,"thread_start_us":100,"threads_started":1}
I20260809 04:45:36.370147 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.365953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:50235 (local address 127.7.171.3:42213)
0809 04:45:36.366161 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.366166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.366490 (+ 324us) server_negotiation.cc:407] Connection header received
0809 04:45:36.366620 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.366623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.366673 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.366758 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:36.367198 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.367895 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.369140 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.369511 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.369589 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.369678 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.369924 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.370004 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 04:45:36.370198 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.365789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:45511 (local address 127.7.171.2:41579)
0809 04:45:36.366157 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.366176 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.366293 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.366612 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.366615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.366637 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:36.366784 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.366798 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.367731 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.367735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.368958 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.368968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.369688 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.369695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.369840 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.369859 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.369956 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.370033 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":40,"thread_start_us":81,"threads_started":1}
I20260809 04:45:36.370205 17037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.365975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:41579 (local address 127.7.171.1:45511)
0809 04:45:36.366223 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.366227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.366244 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:45:36.366435 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.366441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.366502 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.366605 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:36.367019 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.367609 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.369110 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.369540 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.369669 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.369734 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.369996 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.370084 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260809 04:45:36.370918 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636440feff8b470684f4af60730dc79c" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50e73d3e760147c6a0217469e1229e77" is_pre_election: true
I20260809 04:45:36.370920 16627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636440feff8b470684f4af60730dc79c" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b977419893e4504a59f0561b8a8efa8" is_pre_election: true
I20260809 04:45:36.371222 16627 raft_consensus.cc:2454] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 0.
I20260809 04:45:36.371227 16956 raft_consensus.cc:2454] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 0.
I20260809 04:45:36.371758 16713 leader_election.cc:302] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b977419893e4504a59f0561b8a8efa8, 430f845c3f814252b67dcab7c78b0332; no voters:
I20260809 04:45:36.372107 17048 raft_consensus.cc:2781] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:36.372200 17048 raft_consensus.cc:482] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:36.372282 17048 raft_consensus.cc:3037] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:36.373417 17048 raft_consensus.cc:504] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } }
I20260809 04:45:36.373790 17048 leader_election.cc:288] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 election: Requested vote from peers 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213), 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511)
I20260809 04:45:36.374173 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636440feff8b470684f4af60730dc79c" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50e73d3e760147c6a0217469e1229e77"
I20260809 04:45:36.374215 16627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636440feff8b470684f4af60730dc79c" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b977419893e4504a59f0561b8a8efa8"
I20260809 04:45:36.374289 16956 raft_consensus.cc:3037] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:36.374366 16627 raft_consensus.cc:3037] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:36.375478 16627 raft_consensus.cc:2454] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 1.
I20260809 04:45:36.375849 16713 leader_election.cc:302] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b977419893e4504a59f0561b8a8efa8, 430f845c3f814252b67dcab7c78b0332; no voters:
I20260809 04:45:36.375936 16956 raft_consensus.cc:2454] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 1.
I20260809 04:45:36.376084 17048 raft_consensus.cc:2781] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:36.376308 17048 raft_consensus.cc:686] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 1 LEADER]: Becoming Leader. State: Replica: 430f845c3f814252b67dcab7c78b0332, State: Running, Role: LEADER
I20260809 04:45:36.376644 17048 consensus_queue.cc:237] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } }
I20260809 04:45:36.380192 16438 catalog_manager.cc:5495] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 reported cstate change: term changed from 0 to 1, leader changed from <none> to 430f845c3f814252b67dcab7c78b0332 (127.7.171.2). New cstate: current_term: 1 leader_uuid: "430f845c3f814252b67dcab7c78b0332" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } health_report { overall_health: UNKNOWN } } }
I20260809 04:45:36.395001 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.391217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.7.171.2:36701)
0809 04:45:36.391379 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.391383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.391397 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:36.391574 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.391578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.391636 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.391749 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:36.392033 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.392657 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.393538 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.393769 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.393880 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.394173 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:36.394597 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:36.394718 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.394796 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.394856 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:45:36.401136 16627 raft_consensus.cc:1261] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Refusing update from remote peer 430f845c3f814252b67dcab7c78b0332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:36.401188 16956 raft_consensus.cc:1261] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Refusing update from remote peer 430f845c3f814252b67dcab7c78b0332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:36.401858 17048 consensus_queue.cc:1035] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:36.402092 17052 consensus_queue.cc:1035] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:36.410969 17057 mvcc.cc:205] Tried to move back new op lower bound from 7316483016291909632 to 7316483016201715712. Current Snapshot: MvccSnapshot[applied={T|T < 7316483016291909632}]
I20260809 04:45:36.418162 16438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:36.418372 16438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:36.421244 16438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49: Sending DeleteTablet for 3 replicas of tablet 636440feff8b470684f4af60730dc79c
I20260809 04:45:36.422204 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet 636440feff8b470684f4af60730dc79c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:36 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:49828
I20260809 04:45:36.422206 16932 tablet_service.cc:1514] Processing DeleteTablet for tablet 636440feff8b470684f4af60730dc79c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:36 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:58888
I20260809 04:45:36.422502 16605 tablet_service.cc:1514] Processing DeleteTablet for tablet 636440feff8b470684f4af60730dc79c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:36 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:51028
I20260809 04:45:36.422868 17067 tablet_replica.cc:330] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: stopping tablet replica
I20260809 04:45:36.423040 17069 tablet_replica.cc:330] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: stopping tablet replica
I20260809 04:45:36.423183 17068 tablet_replica.cc:330] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: stopping tablet replica
I20260809 04:45:36.423414 17069 raft_consensus.cc:2229] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:36.423766 17069 raft_consensus.cc:2258] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:36.423749 17067 raft_consensus.cc:2229] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:36.423925 17068 raft_consensus.cc:2229] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:36.423929 16438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:36.424439 16438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:36.424538 17067 raft_consensus.cc:2258] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:36.425028 17068 raft_consensus.cc:2258] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:36.426403 17068 ts_tablet_manager.cc:1905] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:36.426415 17067 ts_tablet_manager.cc:1905] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:36.429422 17068 ts_tablet_manager.cc:1918] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:36.429426 17067 ts_tablet_manager.cc:1918] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:36.429536 17068 log.cc:1196] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/636440feff8b470684f4af60730dc79c
I20260809 04:45:36.429667 17067 log.cc:1196] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/636440feff8b470684f4af60730dc79c
I20260809 04:45:36.429896 17068 ts_tablet_manager.cc:1939] T 636440feff8b470684f4af60730dc79c P 430f845c3f814252b67dcab7c78b0332: Deleting consensus metadata
I20260809 04:45:36.430124 17067 ts_tablet_manager.cc:1939] T 636440feff8b470684f4af60730dc79c P 50e73d3e760147c6a0217469e1229e77: Deleting consensus metadata
I20260809 04:45:36.430830 16770 tablet_service.cc:1467] Processing CreateTablet for tablet 5375b1193a5147a7a3a7b7151bf53c68 (DEFAULT_TABLE table=test-table [id=2c42e348cd20434e89102797c1df67c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:36.431110 16770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5375b1193a5147a7a3a7b7151bf53c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.432333 16420 catalog_manager.cc:4841] TS 430f845c3f814252b67dcab7c78b0332 (127.7.171.2:36701): tablet 636440feff8b470684f4af60730dc79c (table test-table [id=36c0e892e9d842da83c39f31c08133c4]) successfully deleted
I20260809 04:45:36.432700 16423 catalog_manager.cc:4841] TS 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213): tablet 636440feff8b470684f4af60730dc79c (table test-table [id=36c0e892e9d842da83c39f31c08133c4]) successfully deleted
I20260809 04:45:36.433020 16932 tablet_service.cc:1467] Processing CreateTablet for tablet 5375b1193a5147a7a3a7b7151bf53c68 (DEFAULT_TABLE table=test-table [id=2c42e348cd20434e89102797c1df67c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:36.433279 16932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5375b1193a5147a7a3a7b7151bf53c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.433408 17044 tablet_bootstrap.cc:492] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332: Bootstrap starting.
I20260809 04:45:36.434433 17044 tablet_bootstrap.cc:654] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.434718 17069 ts_tablet_manager.cc:1905] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:36.435468 16605 tablet_service.cc:1467] Processing CreateTablet for tablet 5375b1193a5147a7a3a7b7151bf53c68 (DEFAULT_TABLE table=test-table [id=2c42e348cd20434e89102797c1df67c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:36.435727 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5375b1193a5147a7a3a7b7151bf53c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.435738 17039 tablet_bootstrap.cc:492] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77: Bootstrap starting.
I20260809 04:45:36.436750 17039 tablet_bootstrap.cc:654] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.438061 17069 ts_tablet_manager.cc:1918] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:36.438225 17040 tablet_bootstrap.cc:492] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8: Bootstrap starting.
I20260809 04:45:36.438226 17069 log.cc:1196] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/wal/wals/636440feff8b470684f4af60730dc79c
I20260809 04:45:36.438444 17044 tablet_bootstrap.cc:492] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332: No bootstrap required, opened a new log
I20260809 04:45:36.438544 17044 ts_tablet_manager.cc:1397] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 04:45:36.438863 17069 ts_tablet_manager.cc:1939] T 636440feff8b470684f4af60730dc79c P 0b977419893e4504a59f0561b8a8efa8: Deleting consensus metadata
I20260809 04:45:36.439157 17044 raft_consensus.cc:348] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.439286 17044 raft_consensus.cc:374] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.439327 17044 raft_consensus.cc:729] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 430f845c3f814252b67dcab7c78b0332, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.439334 17039 tablet_bootstrap.cc:492] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77: No bootstrap required, opened a new log
I20260809 04:45:36.439386 17040 tablet_bootstrap.cc:654] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.439422 17039 ts_tablet_manager.cc:1397] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 04:45:36.439514 17044 consensus_queue.cc:260] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.439973 17039 raft_consensus.cc:348] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.440142 17039 raft_consensus.cc:374] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.440196 17039 raft_consensus.cc:729] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50e73d3e760147c6a0217469e1229e77, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.440363 17044 ts_tablet_manager.cc:1428] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:36.440341 17039 consensus_queue.cc:260] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.440635 17039 ts_tablet_manager.cc:1428] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 04:45:36.440975 16422 catalog_manager.cc:4841] TS 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511): tablet 636440feff8b470684f4af60730dc79c (table test-table [id=36c0e892e9d842da83c39f31c08133c4]) successfully deleted
I20260809 04:45:36.442500 17040 tablet_bootstrap.cc:492] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8: No bootstrap required, opened a new log
I20260809 04:45:36.442616 17040 ts_tablet_manager.cc:1397] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260809 04:45:36.443010 17040 raft_consensus.cc:348] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.443104 17040 raft_consensus.cc:374] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.443126 17040 raft_consensus.cc:729] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b977419893e4504a59f0561b8a8efa8, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.443210 17040 consensus_queue.cc:260] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.443454 17040 ts_tablet_manager.cc:1428] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 04:45:36.450384 17045 raft_consensus.cc:482] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:36.450503 17045 raft_consensus.cc:504] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.451676 17045 leader_election.cc:288] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511), 430f845c3f814252b67dcab7c78b0332 (127.7.171.2:36701)
I20260809 04:45:36.456284 17037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.452402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:46835 (local address 127.7.171.1:45511)
0809 04:45:36.452525 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.452529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.452541 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:36.452590 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.452596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.452636 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.452702 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:36.453108 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.453661 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.455051 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.455654 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.455759 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.455850 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.456025 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.456181 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:45:36.456285 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.451866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:45511 (local address 127.7.171.3:46835)
0809 04:45:36.452326 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.452339 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.452430 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.452858 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.452861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.452878 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:36.452997 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.453002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.453796 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.453801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.454922 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.454933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.455786 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.455790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.455903 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.455930 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.456008 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.456109 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":99,"thread_start_us":150,"threads_started":1}
I20260809 04:45:36.456694 16627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5375b1193a5147a7a3a7b7151bf53c68" candidate_uuid: "50e73d3e760147c6a0217469e1229e77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b977419893e4504a59f0561b8a8efa8" is_pre_election: true
I20260809 04:45:36.456866 16627 raft_consensus.cc:2454] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50e73d3e760147c6a0217469e1229e77 in term 0.
I20260809 04:45:36.457213 16876 leader_election.cc:302] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b977419893e4504a59f0561b8a8efa8, 50e73d3e760147c6a0217469e1229e77; no voters:
I20260809 04:45:36.457427 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.451866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:36701 (local address 127.7.171.3:36609)
0809 04:45:36.451973 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.451989 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.452137 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.452635 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.452638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.452655 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:36.452774 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.452779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.453826 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.453829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.455145 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.455156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.456791 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.456795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.456976 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.457002 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.457099 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.457169 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":41}
I20260809 04:45:36.457492 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.452248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:36609 (local address 127.7.171.2:36701)
0809 04:45:36.452396 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.452402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.452415 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:45:36.452478 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.452481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.452526 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.452608 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:36.452908 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.453688 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.455279 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.455869 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.456021 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.456100 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.457304 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.457378 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:45:36.457566 17045 raft_consensus.cc:2781] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:36.457610 17045 raft_consensus.cc:482] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:36.457645 17045 raft_consensus.cc:3037] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:36.458889 17045 raft_consensus.cc:504] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.459256 17045 leader_election.cc:288] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [CANDIDATE]: Term 1 election: Requested vote from peers 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511), 430f845c3f814252b67dcab7c78b0332 (127.7.171.2:36701)
I20260809 04:45:36.459642 16627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5375b1193a5147a7a3a7b7151bf53c68" candidate_uuid: "50e73d3e760147c6a0217469e1229e77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b977419893e4504a59f0561b8a8efa8"
I20260809 04:45:36.459762 16627 raft_consensus.cc:3037] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:36.459877 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5375b1193a5147a7a3a7b7151bf53c68" candidate_uuid: "50e73d3e760147c6a0217469e1229e77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "430f845c3f814252b67dcab7c78b0332"
I20260809 04:45:36.459846 16793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5375b1193a5147a7a3a7b7151bf53c68" candidate_uuid: "50e73d3e760147c6a0217469e1229e77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "430f845c3f814252b67dcab7c78b0332" is_pre_election: true
I20260809 04:45:36.460107 16793 raft_consensus.cc:2454] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50e73d3e760147c6a0217469e1229e77 in term 0.
I20260809 04:45:36.460901 16627 raft_consensus.cc:2454] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50e73d3e760147c6a0217469e1229e77 in term 1.
I20260809 04:45:36.461167 16876 leader_election.cc:302] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0b977419893e4504a59f0561b8a8efa8, 50e73d3e760147c6a0217469e1229e77; no voters: 430f845c3f814252b67dcab7c78b0332
I20260809 04:45:36.461333 17045 raft_consensus.cc:2781] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:36.461570 17045 raft_consensus.cc:686] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 1 LEADER]: Becoming Leader. State: Replica: 50e73d3e760147c6a0217469e1229e77, State: Running, Role: LEADER
I20260809 04:45:36.461810 17045 consensus_queue.cc:237] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.464308 16439 catalog_manager.cc:5495] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50e73d3e760147c6a0217469e1229e77 (127.7.171.3). New cstate: current_term: 1 leader_uuid: "50e73d3e760147c6a0217469e1229e77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } health_report { overall_health: HEALTHY } } }
Aug 09 04:45:36 dist-test-slave-1zqn krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250735, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.62@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:36.503750 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.494145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.7.171.62:39447)
0809 04:45:36.494304 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.494309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.494523 (+ 214us) server_negotiation.cc:407] Connection header received
0809 04:45:36.494644 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.494646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.494688 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.494759 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:36.495652 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.496327 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.497511 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.497725 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.497809 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.500465 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:36.500468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:36.500469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:36.500489 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:36.501912 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:36.502400 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:36.502405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:36.502407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:36.502483 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:36.502763 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:36.502768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:36.502769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:36.502945 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:36.503061 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.503342 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.503596 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":70}
I20260809 04:45:36.503767 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.494069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:39447 (local address 127.0.0.1:54606)
0809 04:45:36.494417 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.494434 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.494557 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.494814 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.494817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.495398 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:36.495522 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.495530 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.496438 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.496442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.497388 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.497399 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.497847 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.497851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.497944 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.498582 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:36.498597 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:36.500301 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:36.502036 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:36.502049 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:36.502057 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:36.502300 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:36.502587 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:36.502590 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:36.502592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:36.502675 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:36.503052 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:36.503060 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:36.503176 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.503297 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.503546 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":23,"thread_start_us":140,"threads_started":1}
I20260809 04:45:36.508292 16439 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:54606:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:45:36.509032 16439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:36.514087 16605 tablet_service.cc:1467] Processing CreateTablet for tablet 1b826d62158d4eb1aa5bb1ec50917fea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fed6dc6b3adb4e56af6bc2d37f696c78]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:45:36.514210 16770 tablet_service.cc:1467] Processing CreateTablet for tablet 1b826d62158d4eb1aa5bb1ec50917fea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fed6dc6b3adb4e56af6bc2d37f696c78]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:45:36.514544 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b826d62158d4eb1aa5bb1ec50917fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.514591 16770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b826d62158d4eb1aa5bb1ec50917fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.517050 17040 tablet_bootstrap.cc:492] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: Bootstrap starting.
I20260809 04:45:36.517406 17044 tablet_bootstrap.cc:492] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332: Bootstrap starting.
I20260809 04:45:36.518088 17040 tablet_bootstrap.cc:654] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.517957 16932 tablet_service.cc:1467] Processing CreateTablet for tablet 1b826d62158d4eb1aa5bb1ec50917fea (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fed6dc6b3adb4e56af6bc2d37f696c78]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:45:36.518332 16932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b826d62158d4eb1aa5bb1ec50917fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.518404 17044 tablet_bootstrap.cc:654] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.520228 17040 tablet_bootstrap.cc:492] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: No bootstrap required, opened a new log
I20260809 04:45:36.520340 17040 ts_tablet_manager.cc:1397] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260809 04:45:36.520788 17040 raft_consensus.cc:348] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.520907 17040 raft_consensus.cc:374] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.520946 17040 raft_consensus.cc:729] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b977419893e4504a59f0561b8a8efa8, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.521111 17040 consensus_queue.cc:260] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.521395 17039 tablet_bootstrap.cc:492] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77: Bootstrap starting.
I20260809 04:45:36.521423 17040 ts_tablet_manager.cc:1428] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:36.521741 17047 tablet_replica.cc:439] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } }
I20260809 04:45:36.521907 17047 tablet_replica.cc:442] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:36.522337 17044 tablet_bootstrap.cc:492] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332: No bootstrap required, opened a new log
I20260809 04:45:36.522413 17039 tablet_bootstrap.cc:654] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.522418 17044 ts_tablet_manager.cc:1397] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 04:45:36.522830 17044 raft_consensus.cc:348] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.522934 17044 raft_consensus.cc:374] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.522964 17044 raft_consensus.cc:729] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 430f845c3f814252b67dcab7c78b0332, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.523094 17044 consensus_queue.cc:260] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.523342 17052 tablet_replica.cc:439] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } }
I20260809 04:45:36.523452 17052 tablet_replica.cc:442] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:36.524327 17044 ts_tablet_manager.cc:1428] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 04:45:36.524781 17039 tablet_bootstrap.cc:492] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77: No bootstrap required, opened a new log
I20260809 04:45:36.524884 17039 ts_tablet_manager.cc:1397] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 04:45:36.525821 17039 raft_consensus.cc:348] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.525971 17039 raft_consensus.cc:374] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.526175 17039 raft_consensus.cc:729] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50e73d3e760147c6a0217469e1229e77, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.526321 17039 consensus_queue.cc:260] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.526552 17045 tablet_replica.cc:439] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } }
I20260809 04:45:36.526669 17045 tablet_replica.cc:442] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:36.527285 17039 ts_tablet_manager.cc:1428] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:36.730857 17047 raft_consensus.cc:482] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:36.731031 17047 raft_consensus.cc:504] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.732347 17047 leader_election.cc:288] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 430f845c3f814252b67dcab7c78b0332 (127.7.171.2:36701), 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213)
I20260809 04:45:36.736927 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.732731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:49413 (local address 127.7.171.2:36701)
0809 04:45:36.732927 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.732935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.733063 (+ 128us) server_negotiation.cc:407] Connection header received
0809 04:45:36.733359 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.733363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.733432 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.733499 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:36.733900 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.734508 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.735821 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.736361 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.736476 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.736536 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.736724 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.736798 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 04:45:36.736929 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.732956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:51813 (local address 127.7.171.3:42213)
0809 04:45:36.733102 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.733106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.733215 (+ 109us) server_negotiation.cc:407] Connection header received
0809 04:45:36.733407 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.733410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.733453 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.733552 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:36.734031 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.734568 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.735791 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.736255 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.736397 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.736477 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.736676 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.736753 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:45:36.736948 17089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.732593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:36701 (local address 127.7.171.1:49413)
0809 04:45:36.732962 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.732979 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.733166 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.733535 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.733541 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.733559 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:36.733748 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.733797 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.734681 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.734687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.735707 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.735721 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.736482 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.736486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.736591 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.736615 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.736707 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.736773 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":29,"thread_start_us":138,"threads_started":1}
I20260809 04:45:36.736951 17091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.732788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:42213 (local address 127.7.171.1:51813)
0809 04:45:36.733108 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.733121 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.733271 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.733584 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.733587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.733622 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:36.733778 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.733784 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.734744 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.734746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.735621 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.735630 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.736394 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.736399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.736514 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.736546 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.736658 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.736733 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":58,"thread_start_us":122,"threads_started":1}
I20260809 04:45:36.737636 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b826d62158d4eb1aa5bb1ec50917fea" candidate_uuid: "0b977419893e4504a59f0561b8a8efa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "430f845c3f814252b67dcab7c78b0332" is_pre_election: true
I20260809 04:45:36.737779 16792 raft_consensus.cc:2454] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b977419893e4504a59f0561b8a8efa8 in term 0.
I20260809 04:45:36.738204 16546 leader_election.cc:302] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b977419893e4504a59f0561b8a8efa8, 430f845c3f814252b67dcab7c78b0332; no voters:
I20260809 04:45:36.738422 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b826d62158d4eb1aa5bb1ec50917fea" candidate_uuid: "0b977419893e4504a59f0561b8a8efa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50e73d3e760147c6a0217469e1229e77" is_pre_election: true
I20260809 04:45:36.738530 17047 raft_consensus.cc:2781] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:36.738588 17047 raft_consensus.cc:482] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:36.738628 17047 raft_consensus.cc:3037] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:36.738624 16956 raft_consensus.cc:2454] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b977419893e4504a59f0561b8a8efa8 in term 0.
I20260809 04:45:36.740199 17047 raft_consensus.cc:504] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.740546 17047 leader_election.cc:288] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [CANDIDATE]: Term 1 election: Requested vote from peers 430f845c3f814252b67dcab7c78b0332 (127.7.171.2:36701), 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213)
I20260809 04:45:36.740885 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b826d62158d4eb1aa5bb1ec50917fea" candidate_uuid: "0b977419893e4504a59f0561b8a8efa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "430f845c3f814252b67dcab7c78b0332"
I20260809 04:45:36.740885 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b826d62158d4eb1aa5bb1ec50917fea" candidate_uuid: "0b977419893e4504a59f0561b8a8efa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50e73d3e760147c6a0217469e1229e77"
I20260809 04:45:36.741019 16792 raft_consensus.cc:3037] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:36.741091 16956 raft_consensus.cc:3037] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:36.742240 16956 raft_consensus.cc:2454] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b977419893e4504a59f0561b8a8efa8 in term 1.
I20260809 04:45:36.742340 16792 raft_consensus.cc:2454] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b977419893e4504a59f0561b8a8efa8 in term 1.
I20260809 04:45:36.742551 16549 leader_election.cc:302] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b977419893e4504a59f0561b8a8efa8, 50e73d3e760147c6a0217469e1229e77; no voters:
I20260809 04:45:36.742758 17047 raft_consensus.cc:2781] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:36.742995 17047 raft_consensus.cc:686] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [term 1 LEADER]: Becoming Leader. State: Replica: 0b977419893e4504a59f0561b8a8efa8, State: Running, Role: LEADER
I20260809 04:45:36.743263 17047 consensus_queue.cc:237] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.744668 17092 tablet_replica.cc:439] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: TxnStatusTablet state changed. Reason: New leader 0b977419893e4504a59f0561b8a8efa8. Latest consensus state: current_term: 1 leader_uuid: "0b977419893e4504a59f0561b8a8efa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } }
I20260809 04:45:36.744810 17092 tablet_replica.cc:442] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:45:36.745132 17094 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:45:36.745296 17094 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:45:36.745975 16439 catalog_manager.cc:5495] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1). New cstate: current_term: 1 leader_uuid: "0b977419893e4504a59f0561b8a8efa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } health_report { overall_health: UNKNOWN } } }
I20260809 04:45:36.765262 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.761347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:45511 (local address 127.0.0.1:51038)
0809 04:45:36.761518 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.761536 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.761648 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.762044 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.762047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.762055 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:36.762175 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.762181 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.762943 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.762947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.763584 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.763592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.764079 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.764083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.764176 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.764237 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:36.764942 (+ 705us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:36.764961 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.765032 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.765134 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 04:45:36.765262 17037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.761485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.7.171.1:45511)
0809 04:45:36.761625 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.761631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.761650 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:45:36.761859 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.761862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.761921 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.762030 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:36.762298 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.762819 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.763710 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.763945 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.764063 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.764397 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:36.764838 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:36.764969 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.765042 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.765109 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:45:36.768312 16956 raft_consensus.cc:1261] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Refusing update from remote peer 0b977419893e4504a59f0561b8a8efa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:36.768524 16792 raft_consensus.cc:1261] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Refusing update from remote peer 0b977419893e4504a59f0561b8a8efa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:36.768947 17047 consensus_queue.cc:1035] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:36.769146 17092 consensus_queue.cc:1035] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:36.771152 17052 tablet_replica.cc:439] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332: TxnStatusTablet state changed. Reason: New leader 0b977419893e4504a59f0561b8a8efa8. Latest consensus state: current_term: 1 leader_uuid: "0b977419893e4504a59f0561b8a8efa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } }
I20260809 04:45:36.771301 17052 tablet_replica.cc:442] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:36.771704 17045 tablet_replica.cc:439] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77: TxnStatusTablet state changed. Reason: New leader 0b977419893e4504a59f0561b8a8efa8. Latest consensus state: current_term: 1 leader_uuid: "0b977419893e4504a59f0561b8a8efa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } }
I20260809 04:45:36.771828 17045 tablet_replica.cc:442] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:36.774075 17058 mvcc.cc:205] Tried to move back new op lower bound from 7316483017797550080 to 7316483017703251968. Current Snapshot: MvccSnapshot[applied={T|T < 7316483017797550080}]
I20260809 04:45:36.774240 17045 tablet_replica.cc:439] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b977419893e4504a59f0561b8a8efa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } }
I20260809 04:45:36.774327 17045 tablet_replica.cc:442] T 1b826d62158d4eb1aa5bb1ec50917fea P 50e73d3e760147c6a0217469e1229e77: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:36.775092 17056 mvcc.cc:205] Tried to move back new op lower bound from 7316483017797550080 to 7316483017703251968. Current Snapshot: MvccSnapshot[applied={T|T < 7316483017797550080}]
I20260809 04:45:36.775125 17092 tablet_replica.cc:439] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b977419893e4504a59f0561b8a8efa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } }
I20260809 04:45:36.775235 17092 tablet_replica.cc:442] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:45:36.775374 17092 tablet_replica.cc:439] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b977419893e4504a59f0561b8a8efa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } }
I20260809 04:45:36.775460 17092 tablet_replica.cc:442] T 1b826d62158d4eb1aa5bb1ec50917fea P 0b977419893e4504a59f0561b8a8efa8: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:45:36.775629 17052 tablet_replica.cc:439] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b977419893e4504a59f0561b8a8efa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } } }
I20260809 04:45:36.775754 17052 tablet_replica.cc:442] T 1b826d62158d4eb1aa5bb1ec50917fea P 430f845c3f814252b67dcab7c78b0332: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:45:36.783720 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.779446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58902 (local address 127.7.171.3:42213)
0809 04:45:36.779568 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.779572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.779587 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:36.779764 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.779768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.779834 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.779969 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:36.780286 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.781058 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.782038 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.782385 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.782499 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.782740 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:36.783332 (+ 592us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:36.783459 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.783501 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.783564 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 04:45:36.796476 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.791840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:45511 (local address 127.7.171.3:33771)
0809 04:45:36.792267 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.792284 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.792429 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.792806 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.792809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.792817 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:36.792955 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.792961 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.793987 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.793990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.794970 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.794980 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.795587 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.795591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.795689 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.795799 (+ 110us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:36.796148 (+ 349us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:36.796156 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.796224 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.796307 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":99,"threads_started":1}
I20260809 04:45:36.796466 17037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.792026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:33771 (local address 127.7.171.1:45511)
0809 04:45:36.792181 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.792187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.792366 (+ 179us) server_negotiation.cc:407] Connection header received
0809 04:45:36.792621 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.792625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.792694 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.792787 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:36.793082 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.793862 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.795109 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.795445 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.795564 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.795942 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:36.796032 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:36.796143 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.796246 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.796306 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:45:36.804703 16792 raft_consensus.cc:3037] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:36.805995 16792 raft_consensus.cc:1261] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Refusing update from remote peer 50e73d3e760147c6a0217469e1229e77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:36.806352 16627 raft_consensus.cc:1261] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Refusing update from remote peer 50e73d3e760147c6a0217469e1229e77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:36.806849 17074 consensus_queue.cc:1035] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:36.807091 17045 consensus_queue.cc:1035] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:36.833177 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.828363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:38601 (local address 127.7.171.3:42213)
0809 04:45:36.828493 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.828504 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.828840 (+ 336us) server_negotiation.cc:407] Connection header received
0809 04:45:36.829001 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.829004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.829061 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.829154 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:36.829583 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.830384 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.831731 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.832097 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.832209 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.832620 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:36.832737 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:36.832809 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.832950 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.833012 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:45:36.833223 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.828280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:42213 (local address 127.7.171.1:38601)
0809 04:45:36.828730 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:36.828748 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:36.828871 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:36.829183 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:36.829187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:36.829210 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:36.829387 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.829397 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.830550 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.830554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.831597 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:36.831611 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.832298 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.832302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:36.832404 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.832458 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:36.832849 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:36.832859 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 04:45:36.832922 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 04:45:36.833069 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":127,"threads_started":1}
I20260809 04:45:36.868592 16439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:36.868793 16439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:36.871721 16439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49: Sending DeleteTablet for 3 replicas of tablet 5375b1193a5147a7a3a7b7151bf53c68
I20260809 04:45:36.872399 16605 tablet_service.cc:1514] Processing DeleteTablet for tablet 5375b1193a5147a7a3a7b7151bf53c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:36 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:51028
I20260809 04:45:36.872463 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet 5375b1193a5147a7a3a7b7151bf53c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:36 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:49828
I20260809 04:45:36.872632 17069 tablet_replica.cc:330] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8: stopping tablet replica
I20260809 04:45:36.872776 17069 raft_consensus.cc:2229] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:36.872942 17069 raft_consensus.cc:2258] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:36.873140 17068 tablet_replica.cc:330] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332: stopping tablet replica
I20260809 04:45:36.873270 16932 tablet_service.cc:1514] Processing DeleteTablet for tablet 5375b1193a5147a7a3a7b7151bf53c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:36 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:58888
I20260809 04:45:36.873292 17068 raft_consensus.cc:2229] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:36.873456 17067 tablet_replica.cc:330] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77: stopping tablet replica
I20260809 04:45:36.873492 17068 raft_consensus.cc:2258] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:36.873582 17067 raft_consensus.cc:2229] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:36.873870 17067 raft_consensus.cc:2258] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:36.874617 17068 ts_tablet_manager.cc:1905] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:36.874761 17069 ts_tablet_manager.cc:1905] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:36.877768 17069 ts_tablet_manager.cc:1918] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:36.877772 17068 ts_tablet_manager.cc:1918] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:36.877851 17069 log.cc:1196] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/wal/wals/5375b1193a5147a7a3a7b7151bf53c68
I20260809 04:45:36.877856 17068 log.cc:1196] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/5375b1193a5147a7a3a7b7151bf53c68
I20260809 04:45:36.878121 17069 ts_tablet_manager.cc:1939] T 5375b1193a5147a7a3a7b7151bf53c68 P 0b977419893e4504a59f0561b8a8efa8: Deleting consensus metadata
I20260809 04:45:36.878166 17068 ts_tablet_manager.cc:1939] T 5375b1193a5147a7a3a7b7151bf53c68 P 430f845c3f814252b67dcab7c78b0332: Deleting consensus metadata
I20260809 04:45:36.878989 16420 catalog_manager.cc:4841] TS 430f845c3f814252b67dcab7c78b0332 (127.7.171.2:36701): tablet 5375b1193a5147a7a3a7b7151bf53c68 (table test-table [id=2c42e348cd20434e89102797c1df67c2]) successfully deleted
I20260809 04:45:36.879201 16422 catalog_manager.cc:4841] TS 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511): tablet 5375b1193a5147a7a3a7b7151bf53c68 (table test-table [id=2c42e348cd20434e89102797c1df67c2]) successfully deleted
I20260809 04:45:36.879503 17067 ts_tablet_manager.cc:1905] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:36.882390 17067 ts_tablet_manager.cc:1918] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:36.882483 17067 log.cc:1196] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/5375b1193a5147a7a3a7b7151bf53c68
I20260809 04:45:36.882745 17067 ts_tablet_manager.cc:1939] T 5375b1193a5147a7a3a7b7151bf53c68 P 50e73d3e760147c6a0217469e1229e77: Deleting consensus metadata
I20260809 04:45:36.883622 16423 catalog_manager.cc:4841] TS 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213): tablet 5375b1193a5147a7a3a7b7151bf53c68 (table test-table [id=2c42e348cd20434e89102797c1df67c2]) successfully deleted
Aug 09 04:45:36 dist-test-slave-1zqn krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:45:36 dist-test-slave-1zqn krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:36 dist-test-slave-1zqn krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:45:36.920190 16500 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:36.911355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54620 (local address 127.7.171.62:39447)
0809 04:45:36.911511 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.911517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.911727 (+ 210us) server_negotiation.cc:407] Connection header received
0809 04:45:36.911843 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.911848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.911924 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.912088 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:36.913057 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.913853 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.915136 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.915490 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.915616 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.919980 (+ 4364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54620: BlockingRecv error: recv got EOF from 127.0.0.1:54620 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 04:45:36 dist-test-slave-1zqn krb5kdc[16382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM
I20260809 04:45:36.933424 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:36.922193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54632 (local address 127.7.171.62:39447)
0809 04:45:36.922321 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:36.922327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:36.922575 (+ 248us) server_negotiation.cc:407] Connection header received
0809 04:45:36.922736 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:36.922739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:36.922789 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:36.922875 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:36.923705 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.924611 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.925760 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:36.926101 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:36.926205 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:36.929206 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:36.929211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:36.929215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:36.929245 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:36.931169 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:36.931678 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:36.931682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:36.931683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:36.931803 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:36.932261 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:36.932265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:36.932267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:36.932520 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:36.932634 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:36.933042 (+ 408us) server_negotiation.cc:299] Negotiation successful
0809 04:45:36.933172 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:45:36.937356 16439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:36.937849 16439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:36.944033 16605 tablet_service.cc:1467] Processing CreateTablet for tablet 2f8a747a0085471ea77f0e4aaacaa08b (DEFAULT_TABLE table=test-table [id=e3d92902f91f418ab1c793e7cf6100ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:36.944392 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f8a747a0085471ea77f0e4aaacaa08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.944599 16932 tablet_service.cc:1467] Processing CreateTablet for tablet 2f8a747a0085471ea77f0e4aaacaa08b (DEFAULT_TABLE table=test-table [id=e3d92902f91f418ab1c793e7cf6100ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:36.944911 16932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f8a747a0085471ea77f0e4aaacaa08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.947283 17039 tablet_bootstrap.cc:492] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77: Bootstrap starting.
I20260809 04:45:36.948132 17040 tablet_bootstrap.cc:492] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8: Bootstrap starting.
I20260809 04:45:36.948271 17039 tablet_bootstrap.cc:654] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.948939 17040 tablet_bootstrap.cc:654] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.944012 16770 tablet_service.cc:1467] Processing CreateTablet for tablet 2f8a747a0085471ea77f0e4aaacaa08b (DEFAULT_TABLE table=test-table [id=e3d92902f91f418ab1c793e7cf6100ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:36.949340 16770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f8a747a0085471ea77f0e4aaacaa08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:36.951251 17040 tablet_bootstrap.cc:492] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8: No bootstrap required, opened a new log
I20260809 04:45:36.951332 17040 ts_tablet_manager.cc:1397] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:36.951738 17044 tablet_bootstrap.cc:492] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332: Bootstrap starting.
I20260809 04:45:36.951722 17040 raft_consensus.cc:348] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.951856 17040 raft_consensus.cc:374] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.951900 17040 raft_consensus.cc:729] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b977419893e4504a59f0561b8a8efa8, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.952041 17040 consensus_queue.cc:260] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.952325 17040 ts_tablet_manager.cc:1428] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 04:45:36.952826 17044 tablet_bootstrap.cc:654] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:36.954079 17039 tablet_bootstrap.cc:492] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77: No bootstrap required, opened a new log
I20260809 04:45:36.954164 17039 ts_tablet_manager.cc:1397] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 04:45:36.954877 17039 raft_consensus.cc:348] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.954998 17039 raft_consensus.cc:374] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.955030 17039 raft_consensus.cc:729] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50e73d3e760147c6a0217469e1229e77, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.955139 17039 consensus_queue.cc:260] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.955379 17039 ts_tablet_manager.cc:1428] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 04:45:36.956277 17044 tablet_bootstrap.cc:492] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332: No bootstrap required, opened a new log
I20260809 04:45:36.956377 17044 ts_tablet_manager.cc:1397] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 04:45:36.956789 17044 raft_consensus.cc:348] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.956926 17044 raft_consensus.cc:374] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:36.956966 17044 raft_consensus.cc:729] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 430f845c3f814252b67dcab7c78b0332, State: Initialized, Role: FOLLOWER
I20260809 04:45:36.957093 17044 consensus_queue.cc:260] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:36.957350 17044 ts_tablet_manager.cc:1428] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:37.082782 17052 raft_consensus.cc:482] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:37.083025 17052 raft_consensus.cc:504] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.083595 17052 leader_election.cc:288] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511), 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213)
I20260809 04:45:37.084218 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8a747a0085471ea77f0e4aaacaa08b" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50e73d3e760147c6a0217469e1229e77" is_pre_election: true
I20260809 04:45:37.084218 16627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8a747a0085471ea77f0e4aaacaa08b" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b977419893e4504a59f0561b8a8efa8" is_pre_election: true
I20260809 04:45:37.084395 16956 raft_consensus.cc:2454] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 0.
I20260809 04:45:37.084400 16627 raft_consensus.cc:2454] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 0.
I20260809 04:45:37.084798 16714 leader_election.cc:302] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 430f845c3f814252b67dcab7c78b0332, 50e73d3e760147c6a0217469e1229e77; no voters:
I20260809 04:45:37.085021 17052 raft_consensus.cc:2781] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:37.085108 17052 raft_consensus.cc:482] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:37.085166 17052 raft_consensus.cc:3037] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:37.086212 17052 raft_consensus.cc:504] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.086601 17052 leader_election.cc:288] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 election: Requested vote from peers 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511), 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213)
I20260809 04:45:37.087164 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8a747a0085471ea77f0e4aaacaa08b" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50e73d3e760147c6a0217469e1229e77"
I20260809 04:45:37.087280 16956 raft_consensus.cc:3037] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:37.087213 16627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f8a747a0085471ea77f0e4aaacaa08b" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b977419893e4504a59f0561b8a8efa8"
I20260809 04:45:37.087387 16627 raft_consensus.cc:3037] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:37.088389 16956 raft_consensus.cc:2454] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 1.
I20260809 04:45:37.088531 16627 raft_consensus.cc:2454] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 1.
I20260809 04:45:37.088750 16714 leader_election.cc:302] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 430f845c3f814252b67dcab7c78b0332, 50e73d3e760147c6a0217469e1229e77; no voters:
I20260809 04:45:37.088943 17052 raft_consensus.cc:2781] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:37.089133 17052 raft_consensus.cc:686] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 1 LEADER]: Becoming Leader. State: Replica: 430f845c3f814252b67dcab7c78b0332, State: Running, Role: LEADER
I20260809 04:45:37.089301 17052 consensus_queue.cc:237] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.091025 16438 catalog_manager.cc:5495] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 reported cstate change: term changed from 0 to 1, leader changed from <none> to 430f845c3f814252b67dcab7c78b0332 (127.7.171.2). New cstate: current_term: 1 leader_uuid: "430f845c3f814252b67dcab7c78b0332" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } health_report { overall_health: UNKNOWN } } }
I20260809 04:45:37.109558 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:37.105855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49842 (local address 127.7.171.2:36701)
0809 04:45:37.106015 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:37.106020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:37.106070 (+ 50us) server_negotiation.cc:407] Connection header received
0809 04:45:37.106253 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:37.106257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:37.106319 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:37.106417 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:37.106689 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:37.107256 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:37.108378 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:37.108632 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:37.108750 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:37.109054 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:37.109128 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:37.109217 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:37.109317 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 04:45:37.109374 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:45:37.111640 16956 raft_consensus.cc:1261] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Refusing update from remote peer 430f845c3f814252b67dcab7c78b0332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:37.111627 16627 raft_consensus.cc:1261] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Refusing update from remote peer 430f845c3f814252b67dcab7c78b0332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:37.112171 17052 consensus_queue.cc:1035] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:37.112437 17129 consensus_queue.cc:1035] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:37.119663 16438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:37.119838 16438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:37.122381 16438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49: Sending DeleteTablet for 3 replicas of tablet 2f8a747a0085471ea77f0e4aaacaa08b
I20260809 04:45:37.123126 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f8a747a0085471ea77f0e4aaacaa08b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:37 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:49828
I20260809 04:45:37.123117 16603 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f8a747a0085471ea77f0e4aaacaa08b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:37 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:51028
I20260809 04:45:37.123312 16932 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f8a747a0085471ea77f0e4aaacaa08b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:37 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:58888
I20260809 04:45:37.123500 16605 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f8a747a0085471ea77f0e4aaacaa08b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:37 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:51028
I20260809 04:45:37.123515 17067 tablet_replica.cc:330] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77: stopping tablet replica
I20260809 04:45:37.123669 17069 tablet_replica.cc:330] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8: stopping tablet replica
I20260809 04:45:37.123682 17067 raft_consensus.cc:2229] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:37.123785 17069 raft_consensus.cc:2229] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:37.123821 17067 raft_consensus.cc:2258] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:37.123919 17069 raft_consensus.cc:2258] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:37.123847 16438 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 04:45:37.124248 17068 tablet_replica.cc:330] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332: stopping tablet replica
W20260809 04:45:37.124303 16438 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:37.124387 17068 raft_consensus.cc:2229] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:37.124571 17068 raft_consensus.cc:2258] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:37.125255 17068 ts_tablet_manager.cc:1905] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:37.125315 17067 ts_tablet_manager.cc:1905] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:37.125350 17069 ts_tablet_manager.cc:1905] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:37.127246 17068 ts_tablet_manager.cc:1918] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:37.127346 17068 log.cc:1196] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/2f8a747a0085471ea77f0e4aaacaa08b
I20260809 04:45:37.127600 17068 ts_tablet_manager.cc:1939] T 2f8a747a0085471ea77f0e4aaacaa08b P 430f845c3f814252b67dcab7c78b0332: Deleting consensus metadata
I20260809 04:45:37.127717 17069 ts_tablet_manager.cc:1918] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:37.127717 17067 ts_tablet_manager.cc:1918] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:37.127803 17069 log.cc:1196] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-0/wal/wals/2f8a747a0085471ea77f0e4aaacaa08b
I20260809 04:45:37.127803 17067 log.cc:1196] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/2f8a747a0085471ea77f0e4aaacaa08b
I20260809 04:45:37.128100 17067 ts_tablet_manager.cc:1939] T 2f8a747a0085471ea77f0e4aaacaa08b P 50e73d3e760147c6a0217469e1229e77: Deleting consensus metadata
I20260809 04:45:37.128149 17069 ts_tablet_manager.cc:1939] T 2f8a747a0085471ea77f0e4aaacaa08b P 0b977419893e4504a59f0561b8a8efa8: Deleting consensus metadata
I20260809 04:45:37.128409 16420 catalog_manager.cc:4841] TS 430f845c3f814252b67dcab7c78b0332 (127.7.171.2:36701): tablet 2f8a747a0085471ea77f0e4aaacaa08b (table test-table [id=e3d92902f91f418ab1c793e7cf6100ae]) successfully deleted
I20260809 04:45:37.128806 16422 catalog_manager.cc:4841] TS 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511): tablet 2f8a747a0085471ea77f0e4aaacaa08b (table test-table [id=e3d92902f91f418ab1c793e7cf6100ae]) successfully deleted
I20260809 04:45:37.128806 16423 catalog_manager.cc:4841] TS 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213): tablet 2f8a747a0085471ea77f0e4aaacaa08b (table test-table [id=e3d92902f91f418ab1c793e7cf6100ae]) successfully deleted
W20260809 04:45:37.129138 16422 catalog_manager.cc:4808] TS 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511): delete failed for tablet 2f8a747a0085471ea77f0e4aaacaa08b because the tablet was not found. No further retry: Not found: Tablet not found: 2f8a747a0085471ea77f0e4aaacaa08b
I20260809 04:45:37.132344 16605 tablet_service.cc:1467] Processing CreateTablet for tablet a13f11422f9a481fb792be4b29a4b374 (DEFAULT_TABLE table=test-table [id=8361c61571ff4f688ccb6b74a8ececa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:37.132529 16932 tablet_service.cc:1467] Processing CreateTablet for tablet a13f11422f9a481fb792be4b29a4b374 (DEFAULT_TABLE table=test-table [id=8361c61571ff4f688ccb6b74a8ececa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:37.132634 16605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a13f11422f9a481fb792be4b29a4b374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:37.132790 16932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a13f11422f9a481fb792be4b29a4b374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:37.135083 17040 tablet_bootstrap.cc:492] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8: Bootstrap starting.
I20260809 04:45:37.135270 16770 tablet_service.cc:1467] Processing CreateTablet for tablet a13f11422f9a481fb792be4b29a4b374 (DEFAULT_TABLE table=test-table [id=8361c61571ff4f688ccb6b74a8ececa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:37.135540 16770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a13f11422f9a481fb792be4b29a4b374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:37.136188 17040 tablet_bootstrap.cc:654] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:37.136682 17039 tablet_bootstrap.cc:492] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77: Bootstrap starting.
I20260809 04:45:37.137319 17040 tablet_bootstrap.cc:492] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8: No bootstrap required, opened a new log
I20260809 04:45:37.137403 17040 ts_tablet_manager.cc:1397] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 04:45:37.137594 17039 tablet_bootstrap.cc:654] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:37.137638 17044 tablet_bootstrap.cc:492] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332: Bootstrap starting.
I20260809 04:45:37.137905 17040 raft_consensus.cc:348] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.138025 17040 raft_consensus.cc:374] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:37.138067 17040 raft_consensus.cc:729] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b977419893e4504a59f0561b8a8efa8, State: Initialized, Role: FOLLOWER
I20260809 04:45:37.138196 17040 consensus_queue.cc:260] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.138469 17044 tablet_bootstrap.cc:654] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:37.138474 17040 ts_tablet_manager.cc:1428] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 04:45:37.139547 17039 tablet_bootstrap.cc:492] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77: No bootstrap required, opened a new log
I20260809 04:45:37.139642 17039 ts_tablet_manager.cc:1397] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 04:45:37.139720 17044 tablet_bootstrap.cc:492] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332: No bootstrap required, opened a new log
I20260809 04:45:37.139810 17044 ts_tablet_manager.cc:1397] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 04:45:37.140125 17039 raft_consensus.cc:348] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.140230 17039 raft_consensus.cc:374] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:37.140254 17039 raft_consensus.cc:729] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50e73d3e760147c6a0217469e1229e77, State: Initialized, Role: FOLLOWER
I20260809 04:45:37.140244 17044 raft_consensus.cc:348] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.140367 17044 raft_consensus.cc:374] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:37.140401 17044 raft_consensus.cc:729] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 430f845c3f814252b67dcab7c78b0332, State: Initialized, Role: FOLLOWER
I20260809 04:45:37.140398 17039 consensus_queue.cc:260] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.140545 17044 consensus_queue.cc:260] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.140677 17039 ts_tablet_manager.cc:1428] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:37.140833 17044 ts_tablet_manager.cc:1428] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:45:37.214759 17052 raft_consensus.cc:482] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:37.214932 17052 raft_consensus.cc:504] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.215411 17052 leader_election.cc:288] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511), 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213)
I20260809 04:45:37.215898 16627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a13f11422f9a481fb792be4b29a4b374" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b977419893e4504a59f0561b8a8efa8" is_pre_election: true
I20260809 04:45:37.215950 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a13f11422f9a481fb792be4b29a4b374" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50e73d3e760147c6a0217469e1229e77" is_pre_election: true
I20260809 04:45:37.216115 16627 raft_consensus.cc:2454] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 0.
I20260809 04:45:37.216144 16956 raft_consensus.cc:2454] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 0.
I20260809 04:45:37.216504 16714 leader_election.cc:302] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b977419893e4504a59f0561b8a8efa8, 430f845c3f814252b67dcab7c78b0332, 50e73d3e760147c6a0217469e1229e77; no voters:
I20260809 04:45:37.216715 17052 raft_consensus.cc:2781] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:37.216785 17052 raft_consensus.cc:482] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:37.216836 17052 raft_consensus.cc:3037] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:37.218001 17052 raft_consensus.cc:504] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.218360 17052 leader_election.cc:288] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 election: Requested vote from peers 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511), 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213)
I20260809 04:45:37.218735 16627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a13f11422f9a481fb792be4b29a4b374" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b977419893e4504a59f0561b8a8efa8"
I20260809 04:45:37.218750 16956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a13f11422f9a481fb792be4b29a4b374" candidate_uuid: "430f845c3f814252b67dcab7c78b0332" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50e73d3e760147c6a0217469e1229e77"
I20260809 04:45:37.218855 16627 raft_consensus.cc:3037] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:37.218886 16956 raft_consensus.cc:3037] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:37.219869 16956 raft_consensus.cc:2454] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 1.
I20260809 04:45:37.219875 16627 raft_consensus.cc:2454] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 430f845c3f814252b67dcab7c78b0332 in term 1.
I20260809 04:45:37.220250 16714 leader_election.cc:302] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 430f845c3f814252b67dcab7c78b0332, 50e73d3e760147c6a0217469e1229e77; no voters:
I20260809 04:45:37.220511 17052 raft_consensus.cc:2781] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:37.220619 17052 raft_consensus.cc:686] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 1 LEADER]: Becoming Leader. State: Replica: 430f845c3f814252b67dcab7c78b0332, State: Running, Role: LEADER
I20260809 04:45:37.220779 17052 consensus_queue.cc:237] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } }
I20260809 04:45:37.222772 16439 catalog_manager.cc:5495] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 reported cstate change: term changed from 0 to 1, leader changed from <none> to 430f845c3f814252b67dcab7c78b0332 (127.7.171.2). New cstate: current_term: 1 leader_uuid: "430f845c3f814252b67dcab7c78b0332" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "430f845c3f814252b67dcab7c78b0332" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 36701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 } health_report { overall_health: UNKNOWN } } }
I20260809 04:45:37.250775 17037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:37.247709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:33735 (local address 127.7.171.1:45511)
0809 04:45:37.247835 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:37.247839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:37.248028 (+ 189us) server_negotiation.cc:407] Connection header received
0809 04:45:37.248163 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:37.248165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:37.248210 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:37.248269 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:37.248490 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:37.248993 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:37.249818 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:37.250067 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:37.250166 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:37.250405 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:37.250488 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:37.250544 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:37.250620 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 04:45:37.250660 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:45:37.250799 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:37.247610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:45511 (local address 127.7.171.2:33735)
0809 04:45:37.247930 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:37.247943 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:37.248036 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:37.248293 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:37.248295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:37.248301 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:37.248393 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:37.248398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:37.249091 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:37.249094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:37.249704 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:37.249711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:37.250185 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:37.250187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:37.250248 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:37.250292 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:37.250561 (+ 269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:37.250566 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:45:37.250614 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 04:45:37.250679 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":131,"threads_started":1}
I20260809 04:45:37.256358 16627 raft_consensus.cc:1261] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Refusing update from remote peer 430f845c3f814252b67dcab7c78b0332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:37.256731 16956 raft_consensus.cc:1261] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Refusing update from remote peer 430f845c3f814252b67dcab7c78b0332: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:37.256747 17129 consensus_queue.cc:1035] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b977419893e4504a59f0561b8a8efa8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:37.257285 17129 consensus_queue.cc:1035] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50e73d3e760147c6a0217469e1229e77" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:37.273404 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:37.269516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:36701 (local address 127.7.171.1:38639)
0809 04:45:37.269654 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:37.269670 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:37.269749 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:37.270117 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:37.270120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:37.270127 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:45:37.270252 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:37.270259 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:37.271197 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:37.271203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:37.272084 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:37.272096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:37.272712 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:37.272715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:37.272774 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:37.272800 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:45:37.273125 (+ 325us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:45:37.273134 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:45:37.273208 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 04:45:37.273291 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 04:45:37.273403 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:37.269594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:38639 (local address 127.7.171.2:36701)
0809 04:45:37.269708 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:37.269712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:37.269729 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:45:37.269842 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:37.269845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:37.269927 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:37.269997 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:37.270383 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:37.271086 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:37.272207 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:37.272523 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:37.272625 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:37.272953 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:37.273038 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:37.273121 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:37.273218 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 04:45:37.273269 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 04:45:37.295435 16442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:37.295593 16442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:37.297842 16442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd8a8f31194d491cb4202bcae9e1ac49: Sending DeleteTablet for 3 replicas of tablet a13f11422f9a481fb792be4b29a4b374
I20260809 04:45:37.298457 16605 tablet_service.cc:1514] Processing DeleteTablet for tablet a13f11422f9a481fb792be4b29a4b374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:37 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:51028
I20260809 04:45:37.298604 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet a13f11422f9a481fb792be4b29a4b374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:37 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:49828
I20260809 04:45:37.298679 17069 tablet_replica.cc:330] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8: stopping tablet replica
I20260809 04:45:37.298792 17069 raft_consensus.cc:2229] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:37.298807 17068 tablet_replica.cc:330] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332: stopping tablet replica
I20260809 04:45:37.298923 17069 raft_consensus.cc:2258] T a13f11422f9a481fb792be4b29a4b374 P 0b977419893e4504a59f0561b8a8efa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:37.298933 17068 raft_consensus.cc:2229] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:37.299125 17068 raft_consensus.cc:2258] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:37.299248 16932 tablet_service.cc:1514] Processing DeleteTablet for tablet a13f11422f9a481fb792be4b29a4b374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:37 UTC) from {username='oryx', principal='oryx/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:58888
I20260809 04:45:37.299438 17067 tablet_replica.cc:330] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77: stopping tablet replica
I20260809 04:45:37.299465 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 16529
I20260809 04:45:37.299561 17067 raft_consensus.cc:2229] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:37.299688 17067 raft_consensus.cc:2258] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:37.302321 17068 ts_tablet_manager.cc:1905] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:37.305423 17068 ts_tablet_manager.cc:1918] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:37.305529 17068 log.cc:1196] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/a13f11422f9a481fb792be4b29a4b374
I20260809 04:45:37.305531 17067 ts_tablet_manager.cc:1905] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:37.305790 17068 ts_tablet_manager.cc:1939] T a13f11422f9a481fb792be4b29a4b374 P 430f845c3f814252b67dcab7c78b0332: Deleting consensus metadata
I20260809 04:45:37.306650 16420 catalog_manager.cc:4841] TS 430f845c3f814252b67dcab7c78b0332 (127.7.171.2:36701): tablet a13f11422f9a481fb792be4b29a4b374 (table test-table [id=8361c61571ff4f688ccb6b74a8ececa2]) successfully deleted
I20260809 04:45:37.308881 17067 ts_tablet_manager.cc:1918] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:45:37.308974 17067 log.cc:1196] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/a13f11422f9a481fb792be4b29a4b374
I20260809 04:45:37.309234 17067 ts_tablet_manager.cc:1939] T a13f11422f9a481fb792be4b29a4b374 P 50e73d3e760147c6a0217469e1229e77: Deleting consensus metadata
I20260809 04:45:37.310004 16423 catalog_manager.cc:4841] TS 50e73d3e760147c6a0217469e1229e77 (127.7.171.3:42213): tablet a13f11422f9a481fb792be4b29a4b374 (table test-table [id=8361c61571ff4f688ccb6b74a8ececa2]) successfully deleted
W20260809 04:45:37.312817 16422 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:45511 (error 108)
I20260809 04:45:37.313228 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 16691
I20260809 04:45:37.315768 17146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:37.315004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:45511 (local address 127.0.0.1:51040)
0809 04:45:37.315531 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:37.315634 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:45511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":143,"threads_started":1}
W20260809 04:45:37.316211 16422 catalog_manager.cc:4584] TS 0b977419893e4504a59f0561b8a8efa8 (127.7.171.1:45511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a13f11422f9a481fb792be4b29a4b374: Network error: Client connection negotiation failed: client connection to 127.7.171.1:45511: connect: Connection refused (error 111)
I20260809 04:45:37.322940 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 16854
I20260809 04:45:37.332319 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 16397
2026-08-09T04:45:37Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3515 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:37 dist-test-slave-1zqn krb5kdc[17149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:37 dist-test-slave-1zqn krb5kdc[17149](info): set up 2 sockets
Aug 09 04:45:37 dist-test-slave-1zqn krb5kdc[17149](info): commencing operation
krb5kdc: starting...
W20260809 04:45:38.439942 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:38 dist-test-slave-1zqn krb5kdc[17149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:38.576514 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45099
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:45099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:38.689996 17162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:38.690333 17162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:38.690412 17162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:38.694440 17162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:38.694541 17162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:38.694576 17162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:38.694600 17162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:38.694635 17162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:38.700079 17162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:45099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:38.701236 17162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:38.702325 17162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:38.711570 17169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:38.711616 17171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:38.711614 17168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:38.711874 17162 server_base.cc:1034] running on GCE node
I20260809 04:45:38.712522 17162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:38.713617 17162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:38.714789 17162 hybrid_clock.cc:648] HybridClock initialized: now 1786250738714768 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T04:45:38Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1389 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:38 dist-test-slave-1zqn krb5kdc[17176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:38 dist-test-slave-1zqn krb5kdc[17176](info): set up 2 sockets
Aug 09 04:45:38 dist-test-slave-1zqn krb5kdc[17176](info): commencing operation
krb5kdc: starting...
W20260809 04:45:39.829245 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:39 dist-test-slave-1zqn krb5kdc[17176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:39.963486 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45137
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:45137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:40.074918 17189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:40.075251 17189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:40.075335 17189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:40.079350 17189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:40.079450 17189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:40.079489 17189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:40.079527 17189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:40.079562 17189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:40.084895 17189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:45137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:40.086061 17189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:40.087212 17189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:40.095991 17195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:40.096140 17198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:40.095963 17196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:40.096179 17189 server_base.cc:1034] running on GCE node
I20260809 04:45:40.096946 17189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:40.098050 17189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:40.099255 17189 hybrid_clock.cc:648] HybridClock initialized: now 1786250740099239 us; error 37 us; skew 500 ppm
Aug 09 04:45:40 dist-test-slave-1zqn krb5kdc[17176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250740, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:40.102294 17189 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:45:40.103193 17189 webserver.cc:460] Webserver started at http://127.7.171.62:46735/ using document root <none> and password file <none>
I20260809 04:45:40.103783 17189 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:40.103875 17189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:40.104172 17189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:40.106241 17189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "993f0a108d0d40959ed6e14fe0c6c9db"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "7711cf27142741f9486543b27621349d"
server_key_iv: "5b50409b395469629d6f1dd9aa6b5bb9"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.106822 17189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "993f0a108d0d40959ed6e14fe0c6c9db"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "7711cf27142741f9486543b27621349d"
server_key_iv: "5b50409b395469629d6f1dd9aa6b5bb9"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.110729 17189 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:45:40.113214 17206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:40.114445 17189 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:45:40.114593 17189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "993f0a108d0d40959ed6e14fe0c6c9db"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "7711cf27142741f9486543b27621349d"
server_key_iv: "5b50409b395469629d6f1dd9aa6b5bb9"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.114711 17189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:40.130697 17189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:40.133523 17189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:40.133733 17189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:40.140796 17189 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:45137
I20260809 04:45:40.140820 17269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:45137 every 8 connection(s)
I20260809 04:45:40.141747 17189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:40.144491 17270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:40.150007 17270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: Bootstrap starting.
I20260809 04:45:40.150394 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 17189
I20260809 04:45:40.150576 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:40.150858 7852 external_mini_cluster.cc:1428] Setting key 5d3be50d3e0d6bd3624f69985c0b1eb7
I20260809 04:45:40.152878 17270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:40.153654 17270 log.cc:824] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:40.155526 17270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: No bootstrap required, opened a new log
I20260809 04:45:40.159152 17270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } }
I20260809 04:45:40.159416 17270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:40.159502 17270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 993f0a108d0d40959ed6e14fe0c6c9db, State: Initialized, Role: FOLLOWER
I20260809 04:45:40.159950 17270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } }
I20260809 04:45:40.160133 17270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:40.160220 17270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:40.160323 17270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 0 FOLLOWER]: Advancing to term 1
Aug 09 04:45:40 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:40.161584 17270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } }
I20260809 04:45:40.161957 17270 leader_election.cc:302] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 993f0a108d0d40959ed6e14fe0c6c9db; no voters:
I20260809 04:45:40.162364 17270 leader_election.cc:288] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:40.162492 17277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:40.162709 17277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 1 LEADER]: Becoming Leader. State: Replica: 993f0a108d0d40959ed6e14fe0c6c9db, State: Running, Role: LEADER
I20260809 04:45:40.163043 17277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } }
I20260809 04:45:40.163643 17270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:40.164815 17278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } } }
I20260809 04:45:40.164950 17278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:40.165282 17279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 993f0a108d0d40959ed6e14fe0c6c9db. Latest consensus state: current_term: 1 leader_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } } }
I20260809 04:45:40.165375 17279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:40.166302 17287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:40.167629 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:40.152293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50842 (local address 127.7.171.62:45137)
0809 04:45:40.152774 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:40.152791 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:40.152829 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:45:40.153630 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:40.153649 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:40.154058 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:40.154413 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:40.155326 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.156526 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.157497 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.158045 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.158239 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:40.161410 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:40.161415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:40.161428 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:40.161455 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:40.163775 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:40.164380 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:40.164386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:40.164392 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:40.164555 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:40.164923 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:40.164927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:40.164929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:40.165528 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:40.165742 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:40.166513 (+ 771us) server_negotiation.cc:299] Negotiation successful
0809 04:45:40.166839 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":151,"threads_started":1}
I20260809 04:45:40.169837 17287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:40.175719 17287 catalog_manager.cc:1347] Generated new cluster ID: 16c995267a2447a19700daf412119085
I20260809 04:45:40.175793 17287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:40.203737 17287 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:40.205163 17287 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:40.210708 17287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: Generated new TSK 0
I20260809 04:45:40.211532 17287 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:45:40.250859 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45137
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:40.366202 17302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:40.366505 17302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:40.366582 17302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:40.370628 17302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:40.370720 17302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:40.370818 17302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:40.376120 17302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:40.377209 17302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:40.378266 17302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:40.388082 17307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:40.388170 17308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:40.388079 17310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:40.388288 17302 server_base.cc:1034] running on GCE node
I20260809 04:45:40.388860 17302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:40.389509 17302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:40.390725 17302 hybrid_clock.cc:648] HybridClock initialized: now 1786250740390692 us; error 41 us; skew 500 ppm
Aug 09 04:45:40 dist-test-slave-1zqn krb5kdc[17176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250740, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:40.393635 17302 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:45:40.394476 17302 webserver.cc:460] Webserver started at http://127.7.171.1:46373/ using document root <none> and password file <none>
I20260809 04:45:40.395040 17302 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:40.395150 17302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:40.395390 17302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:40.397513 17302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "7dfe6123e9ec4f29aa57a0acbc5121e6"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "4228aa9493b7e048c6f1261e79f52fe3"
server_key_iv: "33b4e400ecb2a50c7a06729dd7023a71"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.398111 17302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "7dfe6123e9ec4f29aa57a0acbc5121e6"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "4228aa9493b7e048c6f1261e79f52fe3"
server_key_iv: "33b4e400ecb2a50c7a06729dd7023a71"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.401862 17302 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:45:40.404279 17321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:40.405441 17302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:40.405572 17302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "7dfe6123e9ec4f29aa57a0acbc5121e6"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "4228aa9493b7e048c6f1261e79f52fe3"
server_key_iv: "33b4e400ecb2a50c7a06729dd7023a71"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.405704 17302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:40.438339 17302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:40.441502 17302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:40.441720 17302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:40.442342 17302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:40.443228 17302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:40.443300 17302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:40.443373 17302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:40.443423 17302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:40.452879 17302 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:36929
I20260809 04:45:40.452932 17452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:36929 every 8 connection(s)
I20260809 04:45:40.453809 17302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:40.458627 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 17302
I20260809 04:45:40.458784 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:40.459116 7852 external_mini_cluster.cc:1428] Setting key 680280beb99dca62ecdb0c3453df05c9
Aug 09 04:45:40 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250740, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:40.468727 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:40.455755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:43491 (local address 127.7.171.62:45137)
0809 04:45:40.455895 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:40.455898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:40.456946 (+ 1048us) server_negotiation.cc:407] Connection header received
0809 04:45:40.457847 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:40.457850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:40.457908 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:40.457990 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:40.459325 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.459987 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.461276 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.461521 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.461644 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:40.464524 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:40.464530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:40.464537 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:40.464570 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:40.466444 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:40.466952 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:40.466958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:40.466963 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:40.467065 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:40.467405 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:40.467411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:40.467414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:40.467708 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:40.467828 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:40.468369 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 04:45:40.468511 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:45:40.469468 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:40.456141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:43491)
0809 04:45:40.456793 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:40.456835 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:40.457640 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:40.458160 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:40.458167 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:40.458514 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:40.459110 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:40.459125 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.460143 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.460146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:40.461150 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:40.461160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.461681 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.461685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:40.461847 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:40.462484 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:40.462499 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:40.464337 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:40.466561 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:40.466566 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:40.466586 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:40.466841 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:40.467169 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:40.467171 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:40.467173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:40.467306 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:40.467820 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:40.467825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:40.468107 (+ 282us) client_negotiation.cc:769] Sending connection context
0809 04:45:40.468288 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 04:45:40.468551 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:45:40.470844 17453 heartbeater.cc:347] Connected to a master server at 127.7.171.62:45137
I20260809 04:45:40.471102 17453 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:40.471611 17453 heartbeater.cc:511] Master 127.7.171.62:45137 requested a full tablet report, sending...
I20260809 04:45:40.473290 17228 ts_manager.cc:194] Registered new tserver with Master: 7dfe6123e9ec4f29aa57a0acbc5121e6 (127.7.171.1:36929)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
I20260809 04:45:40.474581 17228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:43491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:45:40.491494 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45137
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:40.616887 17459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:40.617391 17459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:40.617502 17459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:40.623632 17459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:40.623735 17459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:40.623844 17459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:40.629705 17459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:40.630872 17459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:40.632324 17459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:40.641960 17464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:40.641965 17468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:40.641961 17465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:40.642354 17459 server_base.cc:1034] running on GCE node
I20260809 04:45:40.642771 17459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:40.643358 17459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:40.644512 17459 hybrid_clock.cc:648] HybridClock initialized: now 1786250740644480 us; error 46 us; skew 500 ppm
Aug 09 04:45:40 dist-test-slave-1zqn krb5kdc[17176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250740, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:40.647388 17459 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:45:40.648356 17459 webserver.cc:460] Webserver started at http://127.7.171.2:36825/ using document root <none> and password file <none>
I20260809 04:45:40.648912 17459 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:40.648973 17459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:40.649216 17459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:40.651244 17459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "ea174015e65b440292ca5d87525f0115"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "510b761970678271dc786958ab71bace"
server_key_iv: "cf18dd54a539ebc3f24fc3c807a68d9f"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.651830 17459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "ea174015e65b440292ca5d87525f0115"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "510b761970678271dc786958ab71bace"
server_key_iv: "cf18dd54a539ebc3f24fc3c807a68d9f"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.655660 17459 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 04:45:40.658252 17475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:40.659471 17459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:45:40.659593 17459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "ea174015e65b440292ca5d87525f0115"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "510b761970678271dc786958ab71bace"
server_key_iv: "cf18dd54a539ebc3f24fc3c807a68d9f"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.659695 17459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:40.669772 17459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:40.672623 17459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:40.672820 17459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:40.673394 17459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:40.674198 17459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:40.674247 17459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:40.674317 17459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:40.674355 17459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:40.683669 17459 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:46615
I20260809 04:45:40.683712 17599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:46615 every 8 connection(s)
I20260809 04:45:40.684648 17459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:40.688808 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 17459
I20260809 04:45:40.688902 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:40.689172 7852 external_mini_cluster.cc:1428] Setting key 7b215c335a4da85bf6524372815b90e4
Aug 09 04:45:40 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250740, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:40.699491 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:40.686874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:38059 (local address 127.7.171.62:45137)
0809 04:45:40.687016 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:40.687021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:40.688039 (+ 1018us) server_negotiation.cc:407] Connection header received
0809 04:45:40.689092 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:40.689095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:40.689155 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:40.689232 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:40.690807 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.691335 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.692568 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.692902 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.693027 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:40.695841 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:40.695844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:40.695846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:40.695869 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:40.697383 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:40.697912 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:40.697919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:40.697921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:40.698004 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:40.698366 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:40.698370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:40.698372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:40.698554 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:40.698681 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:40.699194 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 04:45:40.699318 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:45:40.700453 17605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:40.687256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:38059)
0809 04:45:40.687910 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:40.687951 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:40.688824 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:40.689407 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:40.689415 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:40.689899 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:40.690621 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:40.690644 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.691467 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.691471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:40.692443 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:40.692453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.693067 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.693071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:40.693243 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:40.693867 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:40.693883 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:40.695664 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:40.697513 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:40.697519 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:40.697533 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:40.697803 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:40.698101 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:40.698107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:40.698111 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:40.698264 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:40.698681 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:40.698689 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:40.698945 (+ 256us) client_negotiation.cc:769] Sending connection context
0809 04:45:40.699140 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 04:45:40.699468 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":113,"threads_started":1}
I20260809 04:45:40.701665 17601 heartbeater.cc:347] Connected to a master server at 127.7.171.62:45137
I20260809 04:45:40.702028 17601 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
I20260809 04:45:40.702658 17601 heartbeater.cc:511] Master 127.7.171.62:45137 requested a full tablet report, sending...
I20260809 04:45:40.703847 17228 ts_manager.cc:194] Registered new tserver with Master: ea174015e65b440292ca5d87525f0115 (127.7.171.2:46615)
I20260809 04:45:40.704494 17228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:38059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:45:40.719908 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45137
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:40.838653 17608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:40.838905 17608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:40.838984 17608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:40.842995 17608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:40.843060 17608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:40.843148 17608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:40.848431 17608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:40.849435 17608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:40.850524 17608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:40.860320 17616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:40.860232 17620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:40.860229 17617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:40.860354 17608 server_base.cc:1034] running on GCE node
I20260809 04:45:40.861016 17608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:40.861589 17608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:40.862779 17608 hybrid_clock.cc:648] HybridClock initialized: now 1786250740862766 us; error 24 us; skew 500 ppm
Aug 09 04:45:40 dist-test-slave-1zqn krb5kdc[17176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250740, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:40.865877 17608 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:45:40.866715 17608 webserver.cc:460] Webserver started at http://127.7.171.3:41485/ using document root <none> and password file <none>
I20260809 04:45:40.867270 17608 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:40.867331 17608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:40.867570 17608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:40.869690 17608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "4f950cd19eed4bc09634e432af026ee4"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "a8e1be4dd87a82c7f7aa8c51aaa17a86"
server_key_iv: "d2aec2665089077c045cb1b3b63f7a64"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.870270 17608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "4f950cd19eed4bc09634e432af026ee4"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "a8e1be4dd87a82c7f7aa8c51aaa17a86"
server_key_iv: "d2aec2665089077c045cb1b3b63f7a64"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.874104 17608 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 04:45:40.876574 17627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:40.877795 17608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:40.877897 17608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "4f950cd19eed4bc09634e432af026ee4"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "a8e1be4dd87a82c7f7aa8c51aaa17a86"
server_key_iv: "d2aec2665089077c045cb1b3b63f7a64"
server_key_version: "encryptionkey@0"
I20260809 04:45:40.878106 17608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:40.891718 17608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:40.895097 17608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:40.895300 17608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:40.895891 17608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:40.896754 17608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:40.896800 17608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:40.896867 17608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:40.896901 17608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:40.906013 17608 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:45869
I20260809 04:45:40.906049 17758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:45869 every 8 connection(s)
I20260809 04:45:40.907023 17608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
Aug 09 04:45:40 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250740, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:45:40.916973 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 17608
I20260809 04:45:40.917131 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:40.917493 7852 external_mini_cluster.cc:1428] Setting key 82cb9467f250a8eddd80a67b808b50ac
I20260809 04:45:40.921506 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:40.908786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:40801 (local address 127.7.171.62:45137)
0809 04:45:40.908936 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:40.908941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:40.910087 (+ 1146us) server_negotiation.cc:407] Connection header received
0809 04:45:40.911216 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:40.911218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:40.911266 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:40.911340 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:40.912785 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.913301 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.914536 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.914756 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.914833 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:40.917606 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:40.917610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:40.917613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:40.917642 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:40.919541 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:40.920157 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:40.920160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:40.920162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:40.920251 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:40.920555 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:40.920557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:40.920558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:40.920749 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:40.920823 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:40.921285 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 04:45:40.921385 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:45:40.922365 17762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:40.909242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:40801)
0809 04:45:40.909910 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:40.909952 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:40.911007 (+ 1055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:40.911583 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:40.911591 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:40.912017 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:40.912640 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:40.912657 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.913503 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.913506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:40.914426 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:40.914436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:40.914889 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:40.914892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:40.915041 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:40.915735 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:40.915752 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:40.917432 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:40.919669 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:40.919675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:40.919691 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:40.920049 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:40.920321 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:40.920324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:40.920325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:40.920466 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:40.920836 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:40.920841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:40.921072 (+ 231us) client_negotiation.cc:769] Sending connection context
0809 04:45:40.921223 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 04:45:40.921480 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":115,"threads_started":1}
I20260809 04:45:40.923628 17759 heartbeater.cc:347] Connected to a master server at 127.7.171.62:45137
I20260809 04:45:40.923877 17759 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:40.924432 17759 heartbeater.cc:511] Master 127.7.171.62:45137 requested a full tablet report, sending...
I20260809 04:45:40.925447 17228 ts_manager.cc:194] Registered new tserver with Master: 4f950cd19eed4bc09634e432af026ee4 (127.7.171.3:45869)
I20260809 04:45:40.925979 17228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:40801
I20260809 04:45:40.931830 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab.
Entry for principal oryx/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab.
WARNING: no policy specified for oryx/127.7.171.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab.
Entry for principal oryx/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab.
WARNING: no policy specified for oryx/127.7.171.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.171.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab.
Entry for principal oryx/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab.
I20260809 04:45:41.014642 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 17302
I20260809 04:45:41.024493 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 17459
I20260809 04:45:41.032035 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 17608
I20260809 04:45:41.044483 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 17189
I20260809 04:45:41.052789 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45137
--webserver_interface=127.7.171.62
--webserver_port=46735
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:45137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:41.165024 17769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:41.165326 17769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:41.165433 17769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:41.169538 17769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:41.169620 17769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:41.169656 17769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:41.169715 17769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:41.169752 17769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:41.175065 17769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:45137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=46735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:41.176205 17769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:41.177299 17769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:41.186030 17779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:41.186187 17776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:41.186028 17774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:41.186069 17769 server_base.cc:1034] running on GCE node
I20260809 04:45:41.186853 17769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:41.188017 17769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:41.189245 17769 hybrid_clock.cc:648] HybridClock initialized: now 1786250741189233 us; error 30 us; skew 500 ppm
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250741, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:41.192390 17769 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:45:41.193357 17769 webserver.cc:460] Webserver started at http://127.7.171.62:46735/ using document root <none> and password file <none>
I20260809 04:45:41.193962 17769 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:41.194075 17769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:41.198239 17769 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:41.200580 17786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:41.201818 17769 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:41.201967 17769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "993f0a108d0d40959ed6e14fe0c6c9db"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "7711cf27142741f9486543b27621349d"
server_key_iv: "5b50409b395469629d6f1dd9aa6b5bb9"
server_key_version: "encryptionkey@0"
I20260809 04:45:41.202538 17769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:41.226500 17769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:41.230123 17769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:41.230345 17769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:41.237908 17769 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:45137
I20260809 04:45:41.237937 17841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:45137 every 8 connection(s)
I20260809 04:45:41.239004 17769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:41.239674 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 17769
I20260809 04:45:41.240217 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:36929
--local_ip_for_outbound_sockets=127.7.171.1
--tserver_master_addrs=127.7.171.62:45137
--webserver_port=46373
--webserver_interface=127.7.171.1
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.171.1
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:45:41.243695 17842 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:45:41.246403 17842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: Bootstrap starting.
I20260809 04:45:41.270869 17842 log.cc:824] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:41.279394 17842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:45:41.279847 17842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: Bootstrap complete.
I20260809 04:45:41.285769 17842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } }
I20260809 04:45:41.286353 17842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 993f0a108d0d40959ed6e14fe0c6c9db, State: Initialized, Role: FOLLOWER
I20260809 04:45:41.286936 17842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } }
I20260809 04:45:41.287094 17842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:41.287178 17842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:41.287276 17842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:45:41.289959 17842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } }
I20260809 04:45:41.290352 17842 leader_election.cc:302] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 993f0a108d0d40959ed6e14fe0c6c9db; no voters:
I20260809 04:45:41.290710 17842 leader_election.cc:288] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:45:41.290894 17848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:45:41.291203 17848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [term 2 LEADER]: Becoming Leader. State: Replica: 993f0a108d0d40959ed6e14fe0c6c9db, State: Running, Role: LEADER
I20260809 04:45:41.291527 17848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } }
I20260809 04:45:41.292032 17842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:41.293081 17849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } } }
I20260809 04:45:41.293113 17850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 993f0a108d0d40959ed6e14fe0c6c9db. Latest consensus state: current_term: 2 leader_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993f0a108d0d40959ed6e14fe0c6c9db" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45137 } } }
I20260809 04:45:41.293223 17850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:41.293224 17849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:41.297241 17863 catalog_manager.cc:1259] Loaded cluster ID: 16c995267a2447a19700daf412119085
I20260809 04:45:41.297302 17863 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: loading cluster ID for follower catalog manager: success
I20260809 04:45:41.299715 17863 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: acquiring CA information for follower catalog manager: success
I20260809 04:45:41.301131 17863 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:45:41.301270 17857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:41.302129 17857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:41.302564 17857 catalog_manager.cc:1259] Loaded cluster ID: 16c995267a2447a19700daf412119085
I20260809 04:45:41.302624 17857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:41.303247 17857 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:41.303705 17857 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 993f0a108d0d40959ed6e14fe0c6c9db: Loaded TSK: 0
I20260809 04:45:41.304458 17857 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:45:41.373380 17845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:41.373680 17845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:41.373759 17845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:41.377909 17845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:41.377990 17845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:41.378080 17845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:41.383407 17845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.1.keytab
--principal=oryx/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:36929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=46373
--tserver_master_addrs=127.7.171.62:45137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:41.384511 17845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:41.385530 17845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:41.396670 17870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:41.396664 17871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:41.396664 17873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:41.397315 17845 server_base.cc:1034] running on GCE node
I20260809 04:45:41.397773 17845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:41.398628 17845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:41.399855 17845 hybrid_clock.cc:648] HybridClock initialized: now 1786250741399837 us; error 34 us; skew 500 ppm
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250741, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:41.403419 17845 init.cc:377] Logged in from keytab as oryx/127.7.171.1@KRBTEST.COM (short username oryx)
I20260809 04:45:41.404603 17845 webserver.cc:460] Webserver started at http://127.7.171.1:46373/ using document root <none> and password file <none>
I20260809 04:45:41.405349 17845 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:41.405494 17845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:41.410943 17845 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:45:41.413661 17880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:41.415216 17845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:45:41.415354 17845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "7dfe6123e9ec4f29aa57a0acbc5121e6"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "4228aa9493b7e048c6f1261e79f52fe3"
server_key_iv: "33b4e400ecb2a50c7a06729dd7023a71"
server_key_version: "encryptionkey@0"
I20260809 04:45:41.416138 17845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:41.449278 17845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:41.452775 17845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:41.452998 17845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:41.453639 17845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:41.454547 17845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:41.454622 17845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:41.454694 17845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:41.454742 17845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:41.463730 17845 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:36929
I20260809 04:45:41.463754 18009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:36929 every 8 connection(s)
I20260809 04:45:41.464814 17845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:41.469064 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 17845
I20260809 04:45:41.469537 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:46615
--local_ip_for_outbound_sockets=127.7.171.2
--tserver_master_addrs=127.7.171.62:45137
--webserver_port=36825
--webserver_interface=127.7.171.2
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.171.2
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:45:41.478960 18012 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:41.467026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:44045 (local address 127.7.171.62:45137)
0809 04:45:41.467500 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:41.467522 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:41.468026 (+ 504us) server_negotiation.cc:407] Connection header received
0809 04:45:41.469265 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:41.469278 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:41.469587 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:41.470002 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:41.471182 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.472231 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.473411 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.473899 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.474111 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.478154 (+ 4043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:44045: BlockingRecv error: recv got EOF from 127.7.171.1:44045 (error 108)
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":104,"threads_started":1}
I20260809 04:45:41.478972 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:41.467176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:44045)
0809 04:45:41.467859 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:41.467902 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:41.468808 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:41.470094 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:41.470103 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:41.470545 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:41.470990 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.471003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.472371 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.472375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.473273 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.473282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.474154 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.474162 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.474420 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.475096 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:41.475124 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:41.477922 (+ 2798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":142,"threads_started":1}
W20260809 04:45:41.479457 18010 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:45137 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.171.62:45137: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:41.485368 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:41.480004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:53903)
0809 04:45:41.480258 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:41.480272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:41.480387 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:41.480652 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:41.480655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:41.480827 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:41.480929 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.480935 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.481673 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.481678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.482234 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.482241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.482691 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.482695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.482788 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.483297 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:41.483318 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:41.485214 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177}
W20260809 04:45:41.485461 18012 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:41.480097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:53903 (local address 127.7.171.62:45137)
0809 04:45:41.480224 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:41.480228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:41.480342 (+ 114us) server_negotiation.cc:407] Connection header received
0809 04:45:41.480488 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:41.480491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:41.480545 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:41.480635 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:41.481046 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.481559 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.482356 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.482577 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.482687 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.485340 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:53903: BlockingRecv error: recv got EOF from 127.7.171.1:53903 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:41.491719 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:41.486152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:45855)
0809 04:45:41.486317 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:41.486331 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:41.486401 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:41.486632 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:41.486635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:41.486800 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:41.486901 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.486907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.487670 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.487676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.488629 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.488640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.489051 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.489054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.489147 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.489610 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:41.489625 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:41.491563 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99}
W20260809 04:45:41.491930 18012 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:41.486241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:45855 (local address 127.7.171.62:45137)
0809 04:45:41.486358 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:41.486361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:41.486373 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:41.486485 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:41.486487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:41.486523 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:41.486591 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:41.487009 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.487549 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.488742 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.488945 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.489022 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.491795 (+ 2773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:45855: BlockingRecv error: recv got EOF from 127.7.171.1:45855 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 04:45:41.491963 18010 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:45:41.588692 18014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:41.588995 18014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:41.589082 18014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:41.593540 18014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:41.593643 18014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:41.593735 18014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:41.599191 18014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.2.keytab
--principal=oryx/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:46615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=36825
--tserver_master_addrs=127.7.171.62:45137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:41.600410 18014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:41.601465 18014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:41.612159 18019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:41.612218 18022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:41.612151 18020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:41.612357 18014 server_base.cc:1034] running on GCE node
I20260809 04:45:41.612957 18014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:41.613555 18014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:41.614740 18014 hybrid_clock.cc:648] HybridClock initialized: now 1786250741614705 us; error 43 us; skew 500 ppm
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250741, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:41.617619 18014 init.cc:377] Logged in from keytab as oryx/127.7.171.2@KRBTEST.COM (short username oryx)
I20260809 04:45:41.618562 18014 webserver.cc:460] Webserver started at http://127.7.171.2:36825/ using document root <none> and password file <none>
I20260809 04:45:41.619122 18014 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:41.619197 18014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:41.623433 18014 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:45:41.625908 18030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:41.627215 18014 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:41.627353 18014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "ea174015e65b440292ca5d87525f0115"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "510b761970678271dc786958ab71bace"
server_key_iv: "cf18dd54a539ebc3f24fc3c807a68d9f"
server_key_version: "encryptionkey@0"
I20260809 04:45:41.627930 18014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:41.648309 18014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:41.651216 18014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:41.651427 18014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:41.652036 18014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:41.652940 18014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:41.653021 18014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:41.653102 18014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:41.653153 18014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:41.662475 18014 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:46615
I20260809 04:45:41.662508 18160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:46615 every 8 connection(s)
I20260809 04:45:41.663506 18014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:41.667249 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 18014
I20260809 04:45:41.667675 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:45869
--local_ip_for_outbound_sockets=127.7.171.3
--tserver_master_addrs=127.7.171.62:45137
--webserver_port=41485
--webserver_interface=127.7.171.3
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.171.3
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:45:41.675015 18012 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:41.665374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:37391 (local address 127.7.171.62:45137)
0809 04:45:41.665512 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:41.665516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:41.666562 (+ 1046us) server_negotiation.cc:407] Connection header received
0809 04:45:41.667468 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:41.667472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:41.667538 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:41.667658 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:41.668905 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.669558 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.670733 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.671117 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.671242 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.674846 (+ 3604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:37391: BlockingRecv error: recv got EOF from 127.7.171.2:37391 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:45:41.675386 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:41.665773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:37391)
0809 04:45:41.666439 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:41.666480 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:41.667252 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:41.667845 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:41.667851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:41.668283 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:41.668747 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.668761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.669721 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.669724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.670596 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.670606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.671273 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.671276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.671412 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.672034 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:41.672099 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:41.674547 (+ 2448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":118,"threads_started":1}
W20260809 04:45:41.676007 18161 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:45137 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.171.62:45137: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:41.682044 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:41.676495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:36483)
0809 04:45:41.676620 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:41.676635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:41.676713 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:41.677018 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:41.677021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:41.677159 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:41.677250 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.677254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.678343 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.678346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.678885 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.678893 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.679527 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.679529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.679608 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.680114 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:41.680128 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:41.681917 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:45:41.682155 18012 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:41.676636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:36483 (local address 127.7.171.62:45137)
0809 04:45:41.676776 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:41.676781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:41.676796 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:41.676840 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:41.676844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:41.676897 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:41.676987 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:41.677365 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.678223 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.679000 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.679400 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.679523 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.682035 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:36483: BlockingRecv error: recv got EOF from 127.7.171.2:36483 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:41.688295 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:41.682761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:32827)
0809 04:45:41.682903 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:41.682916 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:41.682978 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:41.683200 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:41.683203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:41.683361 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:41.683465 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.683470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.684186 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.684189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.685044 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.685053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.685455 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.685459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.685539 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.685965 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:41.685984 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:41.688153 (+ 2169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86}
W20260809 04:45:41.688370 18012 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:41.682855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:32827 (local address 127.7.171.62:45137)
0809 04:45:41.682964 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:41.682967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:41.682979 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:41.683055 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:41.683057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:41.683098 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:41.683162 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:41.683575 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.684034 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.685155 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.685352 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.685429 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.688259 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:32827: BlockingRecv error: recv got EOF from 127.7.171.2:32827 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 04:45:41.688597 18161 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:45:41.784991 18164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:41.785303 18164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:41.785383 18164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:41.789402 18164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:41.789510 18164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:41.789793 18164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:41.795975 18164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/krb5kdc/oryx_127.7.171.3.keytab
--principal=oryx/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:45869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=41485
--tserver_master_addrs=127.7.171.62:45137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:41.797125 18164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:41.798142 18164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:41.808406 18174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:41.808413 18171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:41.808377 18170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:41.808702 18164 server_base.cc:1034] running on GCE node
I20260809 04:45:41.809082 18164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:41.809726 18164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:41.810905 18164 hybrid_clock.cc:648] HybridClock initialized: now 1786250741810873 us; error 44 us; skew 500 ppm
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250741, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:41.813800 18164 init.cc:377] Logged in from keytab as oryx/127.7.171.3@KRBTEST.COM (short username oryx)
I20260809 04:45:41.814743 18164 webserver.cc:460] Webserver started at http://127.7.171.3:41485/ using document root <none> and password file <none>
I20260809 04:45:41.815301 18164 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:41.815376 18164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:41.819268 18164 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:45:41.821422 18182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:41.822717 18164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:45:41.822811 18164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "4f950cd19eed4bc09634e432af026ee4"
format_stamp: "Formatted at 2026-08-09 04:45:40 on dist-test-slave-1zqn"
server_key: "a8e1be4dd87a82c7f7aa8c51aaa17a86"
server_key_iv: "d2aec2665089077c045cb1b3b63f7a64"
server_key_version: "encryptionkey@0"
I20260809 04:45:41.823372 18164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:41.853863 18164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:41.856829 18164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:41.856985 18164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:41.857547 18164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:41.858355 18164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:41.858402 18164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:41.858443 18164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:41.858458 18164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:41.867393 18164 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:45869
I20260809 04:45:41.867424 18309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:45869 every 8 connection(s)
I20260809 04:45:41.868476 18164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:41.875921 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 18164
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:45:41.879208 18012 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:41.870448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:44079 (local address 127.7.171.62:45137)
0809 04:45:41.870644 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:41.870649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:41.871550 (+ 901us) server_negotiation.cc:407] Connection header received
0809 04:45:41.872396 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:41.872398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:41.872447 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:41.872521 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:41.873668 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.874177 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.875288 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.875508 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.875597 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.879086 (+ 3489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:44079: BlockingRecv error: recv got EOF from 127.7.171.3:44079 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:45:41.879859 18312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:41.870745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:44079)
0809 04:45:41.871405 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:41.871446 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:41.872214 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:41.872693 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:41.872699 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:41.873050 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:41.873539 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.873553 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.874283 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.874286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.875158 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.875167 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.875646 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.875649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.875785 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.876487 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:41.876504 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:41.878981 (+ 2477us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":115,"threads_started":1}
W20260809 04:45:41.880403 18310 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:45137 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.171.62:45137: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:41.886324 18312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:41.880865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:59655)
0809 04:45:41.881040 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:41.881054 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:41.881157 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:41.881391 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:41.881395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:41.881574 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:41.881696 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.881705 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.882370 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.882374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.883255 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.883269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.883730 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.883734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.883831 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.884269 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:41.884288 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:41.886192 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102}
W20260809 04:45:41.886466 18012 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:41.880939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:59655 (local address 127.7.171.62:45137)
0809 04:45:41.881048 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:41.881051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:41.881141 (+ 90us) server_negotiation.cc:407] Connection header received
0809 04:45:41.881230 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:41.881234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:41.881281 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:41.881360 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:41.881816 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.882286 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.883410 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.883614 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.883693 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.886333 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:59655: BlockingRecv error: recv got EOF from 127.7.171.3:59655 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 04:45:41.886904 18314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:41.877005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41542 (local address 127.7.171.62:45137)
0809 04:45:41.877731 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:41.877734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:41.877746 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:41.877776 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:41.877778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:41.877814 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:41.877918 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:41.878731 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.879253 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.880350 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.880682 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.880765 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.882375 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:41.882382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:41.882402 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:41.882433 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:41.884471 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:41.884984 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:41.884989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:41.884994 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:41.885121 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:41.885442 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:41.885447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:41.885450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:41.885907 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:41.886131 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:41.886391 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 04:45:41.886799 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":108,"threads_started":1}
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:41 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:41.892303 18312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:41.887129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:33825)
0809 04:45:41.887237 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:41.887249 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:41.887340 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:41.887723 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:41.887725 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:41.887846 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:41.887948 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.887955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.888697 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.888701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.889396 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:41.889407 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.889909 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.889913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:41.890001 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.890451 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:41.890463 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:41.892210 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 04:45:41.892477 18314 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:41.887381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:33825 (local address 127.7.171.62:45137)
0809 04:45:41.887532 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:41.887537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:41.887549 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:41.887578 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:41.887581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:41.887617 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:41.887694 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:41.888098 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.888579 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.889513 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:41.889787 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:41.889897 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:41.892327 (+ 2430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:33825: BlockingRecv error: recv got EOF from 127.7.171.3:33825 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260809 04:45:41.892524 18310 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 04:45:42 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:42 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:42.498611 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:42.492656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:44907)
0809 04:45:42.492983 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:42.493002 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:42.493186 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:42.493476 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:42.493478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:42.493699 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:42.493823 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:42.493828 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.494775 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.494777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:42.495356 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:42.495364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.495807 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.495809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:42.495882 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:42.496466 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:42.496478 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:42.498483 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":78,"threads_started":1}
W20260809 04:45:42.498814 18316 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:42.492736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:44907 (local address 127.7.171.62:45137)
0809 04:45:42.493151 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:42.493157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:42.493176 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:45:42.493308 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:42.493310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:42.493358 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:42.493454 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:42.494010 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.494675 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.495472 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.495703 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.495786 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:42.498661 (+ 2875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:44907: BlockingRecv error: recv got EOF from 127.7.171.1:44907 (error 108)
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":98,"threads_started":1}
Aug 09 04:45:42 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:42 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:42.695464 18318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:42.689365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:46297)
0809 04:45:42.689708 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:42.689722 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:42.689865 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:42.690158 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:42.690161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:42.690379 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:42.690478 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:42.690484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.691271 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.691277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:42.691875 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:42.691883 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.692429 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.692433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:42.692516 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:42.693280 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:42.693292 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:42.695331 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":97,"threads_started":1}
W20260809 04:45:42.695772 18316 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:42.689473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:46297 (local address 127.7.171.62:45137)
0809 04:45:42.689649 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:42.689654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:42.689801 (+ 147us) server_negotiation.cc:407] Connection header received
0809 04:45:42.689975 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:42.689985 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:42.690032 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:42.690142 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:42.690618 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.691156 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.692001 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.692307 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.692424 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:42.695497 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:46297: BlockingRecv error: recv got EOF from 127.7.171.2:46297 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 04:45:42 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:42 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:42.899080 18319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:42.893213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:56185)
0809 04:45:42.893499 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:42.893512 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:42.893656 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:42.893945 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:42.893949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:42.894238 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:42.894350 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:42.894357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.895133 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.895139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:42.895732 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:42.895740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.896272 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.896276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:42.896364 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:42.896957 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:42.896971 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:42.898889 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":84,"threads_started":1}
W20260809 04:45:42.899147 18316 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:42.893219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:56185 (local address 127.7.171.62:45137)
0809 04:45:42.893383 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:42.893389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:42.893593 (+ 204us) server_negotiation.cc:407] Connection header received
0809 04:45:42.893767 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:42.893769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:42.893828 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:42.893917 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:42.894471 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.895013 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.895854 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:42.896156 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:42.896266 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:42.899018 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:56185: BlockingRecv error: recv got EOF from 127.7.171.3:56185 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 04:45:43 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:43 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:43.505429 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:43.499548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:34815)
0809 04:45:43.499842 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:43.499856 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:43.499957 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:43.500364 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:43.500368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:43.500562 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:43.500669 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:43.500675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.501438 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.501442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:43.502006 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:43.502014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.502439 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.502443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:43.502510 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:43.503086 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:43.503098 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:43.505288 (+ 2190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":96,"threads_started":1}
W20260809 04:45:43.505626 18321 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:43.499615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:34815 (local address 127.7.171.62:45137)
0809 04:45:43.499855 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:43.499858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:43.499932 (+ 74us) server_negotiation.cc:407] Connection header received
0809 04:45:43.500162 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:43.500165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:43.500212 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:43.500289 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:43.500787 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.501329 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.502119 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.502333 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.502407 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:43.505458 (+ 3051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:34815: BlockingRecv error: recv got EOF from 127.7.171.1:34815 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":89,"threads_started":1}
Aug 09 04:45:43 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:43 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:43.702006 18322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:43.696335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:54919)
0809 04:45:43.696684 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:43.696703 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:43.696855 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:43.697160 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:43.697163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:43.697380 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:43.697474 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:43.697479 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.698248 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.698254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:43.698828 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:43.698836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.699309 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.699312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:43.699409 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:43.699973 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:43.699985 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:43.701867 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":90,"threads_started":1}
W20260809 04:45:43.702131 18321 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:43.696460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:54919 (local address 127.7.171.62:45137)
0809 04:45:43.696660 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:43.696666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:43.696788 (+ 122us) server_negotiation.cc:407] Connection header received
0809 04:45:43.696973 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:43.696978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:43.697030 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:43.697133 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:43.697603 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.698136 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.698952 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.699186 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.699292 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:43.702008 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:54919: BlockingRecv error: recv got EOF from 127.7.171.2:54919 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 04:45:43 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:43 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:43.911777 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:43.900099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:44461)
0809 04:45:43.900364 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:43.900379 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:43.900485 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:43.900762 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:43.900765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:43.900948 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:43.901045 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:43.901050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.901798 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.901801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:43.902366 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:43.902374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.902785 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.902787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:43.902849 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:43.909608 (+ 6759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:43.909624 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:43.911617 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":87,"threads_started":1}
W20260809 04:45:43.911957 18321 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:43.900286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:44461 (local address 127.7.171.62:45137)
0809 04:45:43.900432 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:43.900436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:43.900451 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:43.900589 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:43.900591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:43.900642 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:43.900719 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:43.901177 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.901701 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.902473 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:43.902686 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:43.902762 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:43.911819 (+ 9057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:44461: BlockingRecv error: recv got EOF from 127.7.171.3:44461 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 04:45:44 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:44 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:44.512239 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:44.506317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:52059)
0809 04:45:44.506621 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:44.506658 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:44.506783 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:44.507142 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:44.507146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:44.507359 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:44.507463 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:44.507468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.508329 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.508335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:44.508912 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:44.508919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.509414 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.509418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:44.509507 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:44.510080 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:44.510094 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:44.512006 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":102,"threads_started":1}
W20260809 04:45:44.512434 18325 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:44.506371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:52059 (local address 127.7.171.62:45137)
0809 04:45:44.506671 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:44.506677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:44.506727 (+ 50us) server_negotiation.cc:407] Connection header received
0809 04:45:44.506910 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:44.506913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:44.506977 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:44.507108 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:44.507591 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.508223 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.509036 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.509298 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.509410 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:44.512262 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:52059: BlockingRecv error: recv got EOF from 127.7.171.1:52059 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":84,"threads_started":1}
Aug 09 04:45:44 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:44 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:44.708763 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:44.702928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:41515)
0809 04:45:44.703211 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:44.703235 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:44.703339 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:44.703605 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:44.703608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:44.703795 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:44.703896 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:44.703901 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.704680 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.704683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:44.705411 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:44.705422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.705983 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.705986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:44.706085 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:44.706647 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:44.706659 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:44.708621 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":107,"threads_started":1}
W20260809 04:45:44.708874 18325 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:44.703065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:41515 (local address 127.7.171.62:45137)
0809 04:45:44.703225 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:44.703229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:44.703308 (+ 79us) server_negotiation.cc:407] Connection header received
0809 04:45:44.703421 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:44.703423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:44.703467 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:44.703543 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:44.704002 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.704537 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.705533 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.705829 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.705962 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:44.708755 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:41515: BlockingRecv error: recv got EOF from 127.7.171.2:41515 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 04:45:44 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:44 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:44.918733 18327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:44.912849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:58647)
0809 04:45:44.913148 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:44.913168 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:44.913315 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:44.913641 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:44.913644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:44.913856 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:44.913949 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:44.913955 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.914838 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.914844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:44.915406 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:44.915414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.915881 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.915885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:44.915973 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:44.916532 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:44.916545 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:44.918506 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":89,"threads_started":1}
W20260809 04:45:44.918763 18325 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:44.912924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:58647 (local address 127.7.171.62:45137)
0809 04:45:44.913114 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:44.913119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:44.913250 (+ 131us) server_negotiation.cc:407] Connection header received
0809 04:45:44.913431 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:44.913434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:44.913490 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:44.913583 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:44.914104 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.914727 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.915533 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:44.915762 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:44.915874 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:44.918654 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:58647: BlockingRecv error: recv got EOF from 127.7.171.3:58647 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 04:45:45 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:45 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:45.518661 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:45.512987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:60397)
0809 04:45:45.513305 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:45.513319 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:45.513411 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:45.513696 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:45.513698 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:45.513885 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:45.513998 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:45.514003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.514768 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.514771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:45.515329 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:45.515337 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.515848 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.515851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:45.515916 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:45.516563 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:45.516577 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:45.518468 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":91,"threads_started":1}
W20260809 04:45:45.518776 18329 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:45.513115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:60397 (local address 127.7.171.62:45137)
0809 04:45:45.513389 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:45.513392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:45.513406 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:45.513507 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:45.513509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:45.513552 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:45.513638 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:45.514142 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.514669 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.515473 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.515745 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.515824 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:45.518619 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:60397: BlockingRecv error: recv got EOF from 127.7.171.1:60397 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
Aug 09 04:45:45 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:45 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:45.715075 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:45.709522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:51399)
0809 04:45:45.709871 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:45.709884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:45.709986 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:45.710237 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:45.710239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:45.710426 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:45.710527 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:45.710531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.711256 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.711259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:45.711822 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:45.711829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.712307 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.712309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:45.712373 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:45.712904 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:45.712916 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:45.714895 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":116,"threads_started":1}
W20260809 04:45:45.715227 18329 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:45.709651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:51399 (local address 127.7.171.62:45137)
0809 04:45:45.709799 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:45.709803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:45.709954 (+ 151us) server_negotiation.cc:407] Connection header received
0809 04:45:45.710071 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:45.710073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:45.710123 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:45.710194 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:45.710652 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.711162 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.711925 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.712208 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.712287 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:45.715107 (+ 2820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:51399: BlockingRecv error: recv got EOF from 127.7.171.2:51399 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 04:45:45 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:45 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:45.925119 18331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:45.919589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:36455)
0809 04:45:45.919878 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:45.919893 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:45.919998 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:45.920277 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:45.920279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:45.920466 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:45.920565 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:45.920570 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.921299 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.921301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:45.921932 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:45.921942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.922368 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.922371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:45.922431 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:45.923067 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:45.923081 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:45.924976 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":77,"threads_started":1}
W20260809 04:45:45.925248 18329 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:45.919735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:36455 (local address 127.7.171.62:45137)
0809 04:45:45.919905 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:45.919909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:45.919965 (+ 56us) server_negotiation.cc:407] Connection header received
0809 04:45:45.920107 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:45.920109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:45.920161 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:45.920235 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:45.920685 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.921202 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.922039 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:45.922265 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:45.922349 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:45.925123 (+ 2774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:36455: BlockingRecv error: recv got EOF from 127.7.171.3:36455 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 04:45:46 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:46 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:46.526113 18332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:46.519564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:52233)
0809 04:45:46.519869 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:46.519884 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:46.519977 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:46.520252 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:46.520255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:46.520448 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:46.520549 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:46.520554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.521410 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.521412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:46.522047 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:46.522056 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.522674 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.522678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:46.522753 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:46.523433 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:46.523447 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:46.525970 (+ 2523us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":104,"threads_started":1}
W20260809 04:45:46.526248 18333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:46.519717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:52233 (local address 127.7.171.62:45137)
0809 04:45:46.519962 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:46.519965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:46.519980 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:46.520076 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:46.520078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:46.520126 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:46.520211 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:46.520667 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.521310 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.522206 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.522530 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.522631 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:46.526115 (+ 3484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:52233: BlockingRecv error: recv got EOF from 127.7.171.1:52233 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
Aug 09 04:45:46 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:46 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:46.722338 18334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:46.716220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:50207)
0809 04:45:46.716617 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:46.716634 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:46.716791 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:46.717083 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:46.717086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:46.717312 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:46.717416 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:46.717428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.718258 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.718264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:46.718900 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:46.718908 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.719392 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.719396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:46.719500 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:46.720166 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:46.720178 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:46.722217 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":119,"threads_started":1}
W20260809 04:45:46.722525 18333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:46.716439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:50207 (local address 127.7.171.62:45137)
0809 04:45:46.716620 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:46.716627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:46.716727 (+ 100us) server_negotiation.cc:407] Connection header received
0809 04:45:46.716897 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:46.716901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:46.716960 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:46.717058 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:46.717543 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.718141 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.719035 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.719263 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.719370 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:46.722368 (+ 2998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:50207: BlockingRecv error: recv got EOF from 127.7.171.2:50207 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 04:45:46 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:46 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:46.932502 18336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:46.926064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:48103)
0809 04:45:46.926383 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:46.926397 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:46.926537 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:46.927103 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:46.927106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:46.927400 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:46.927529 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:46.927545 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.928479 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.928482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:46.929225 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:46.929235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.929669 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.929672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:46.929739 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:46.930391 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:46.930404 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:46.932382 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":103,"threads_started":1}
W20260809 04:45:46.932727 18333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:46.926136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:48103 (local address 127.7.171.62:45137)
0809 04:45:46.926291 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:46.926296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:46.926471 (+ 175us) server_negotiation.cc:407] Connection header received
0809 04:45:46.926637 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:46.926640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:46.926687 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:46.926758 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:46.927665 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.928367 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.929334 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:46.929548 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:46.929652 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:46.932608 (+ 2956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:48103: BlockingRecv error: recv got EOF from 127.7.171.3:48103 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 04:45:47 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:47 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:47.533161 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:47.527104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:43183)
0809 04:45:47.527442 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:47.527464 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:47.527673 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:47.528041 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:47.528044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:47.528363 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:47.528477 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:47.528490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.529249 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.529252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:47.529819 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:47.529827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.530250 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.530252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:47.530315 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:47.530799 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:47.530811 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:47.533014 (+ 2203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":84,"threads_started":1}
W20260809 04:45:47.533317 18339 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:47.527217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:43183 (local address 127.7.171.62:45137)
0809 04:45:47.527459 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:47.527465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:47.527546 (+ 81us) server_negotiation.cc:407] Connection header received
0809 04:45:47.527771 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:47.527774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:47.527820 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:47.527897 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:47.528605 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.529147 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.529920 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.530147 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.530224 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:47.533173 (+ 2949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:43183: BlockingRecv error: recv got EOF from 127.7.171.1:43183 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":60,"threads_started":1}
Aug 09 04:45:47 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:47 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:47.729035 18340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:47.723156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:50045)
0809 04:45:47.723473 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:47.723487 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:47.723594 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:47.723823 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:47.723825 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:47.724014 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:47.724302 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:47.724309 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.725037 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.725039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:47.725782 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:47.725800 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.726320 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.726324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:47.726416 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:47.726988 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:47.727000 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:47.728918 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":131,"threads_started":1}
W20260809 04:45:47.729182 18339 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:47.723208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:50045 (local address 127.7.171.62:45137)
0809 04:45:47.723373 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:47.723378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:47.723545 (+ 167us) server_negotiation.cc:407] Connection header received
0809 04:45:47.723666 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:47.723668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:47.723714 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:47.723784 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:47.724413 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.724940 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.725891 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.726196 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.726303 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:47.729063 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:50045: BlockingRecv error: recv got EOF from 127.7.171.2:50045 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 04:45:47 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:47 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:47.939014 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:47.933288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:42179)
0809 04:45:47.933561 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:47.933577 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:47.933681 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:47.933925 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:47.933928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:47.934130 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:47.934228 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:47.934233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.934962 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.934965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:47.935539 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:47.935547 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.936128 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.936132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:47.936251 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:47.936924 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:47.936942 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:47.938891 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":73,"threads_started":1}
W20260809 04:45:47.939172 18339 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:47.933426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:42179 (local address 127.7.171.62:45137)
0809 04:45:47.933593 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:47.933597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:47.933643 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:45:47.933764 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:47.933766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:47.933811 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:47.933882 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:47.934325 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.934868 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.935669 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:47.935959 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:47.936069 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:47.939030 (+ 2961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:42179: BlockingRecv error: recv got EOF from 127.7.171.3:42179 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 04:45:48 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:48 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:48.540691 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:48.534179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:42219)
0809 04:45:48.534547 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:48.534561 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:48.534684 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:48.534945 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:48.534948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:48.535154 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:48.535251 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:48.535256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.536473 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.536477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:48.537215 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:48.537227 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.537657 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.537660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:48.537725 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:48.538292 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:48.538304 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:48.540493 (+ 2189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":126,"threads_started":1}
W20260809 04:45:48.540786 18343 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:48.534252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:42219 (local address 127.7.171.62:45137)
0809 04:45:48.534568 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:48.534572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:48.534660 (+ 88us) server_negotiation.cc:407] Connection header received
0809 04:45:48.534772 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:48.534774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:48.534821 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:48.534905 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:48.535525 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.536348 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.537329 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.537547 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.537635 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:48.540643 (+ 3008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:42219: BlockingRecv error: recv got EOF from 127.7.171.1:42219 (error 108)
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":131,"threads_started":1}
Aug 09 04:45:48 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:48 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:48.735630 18344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:48.729908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:37277)
0809 04:45:48.730233 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:48.730247 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:48.730390 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:48.730627 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:48.730629 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:48.730819 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:48.730917 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:48.730921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.731739 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.731741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:48.732411 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:48.732420 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.732853 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.732856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:48.732916 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:48.733425 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:48.733437 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:48.735494 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":107,"threads_started":1}
W20260809 04:45:48.735970 18343 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:48.729980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:37277 (local address 127.7.171.62:45137)
0809 04:45:48.730167 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:48.730173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:48.730324 (+ 151us) server_negotiation.cc:407] Connection header received
0809 04:45:48.730477 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:48.730480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:48.730530 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:48.730603 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:48.731058 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.731639 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.732516 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.732755 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.732835 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:48.735688 (+ 2853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:37277: BlockingRecv error: recv got EOF from 127.7.171.2:37277 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 04:45:48 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:48 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:48.948498 18346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:48.939849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:33755)
0809 04:45:48.940188 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:48.940252 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:48.940378 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:48.940636 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:48.940639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:48.940839 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:48.940997 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:48.941005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.941755 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.941758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:48.942338 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:48.942348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.942807 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.942810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:48.942884 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:48.946422 (+ 3538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:48.946439 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:48.948377 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":79,"threads_started":1}
W20260809 04:45:48.948876 18343 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:48.939900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:33755 (local address 127.7.171.62:45137)
0809 04:45:48.940044 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:48.940119 (+ 75us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:48.940330 (+ 211us) server_negotiation.cc:407] Connection header received
0809 04:45:48.940447 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:48.940449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:48.940499 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:48.940584 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:48.941118 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.941639 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.942453 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:48.942681 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:48.942764 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:48.948775 (+ 6011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:33755: BlockingRecv error: recv got EOF from 127.7.171.3:33755 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 04:45:49 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:49 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:49.548094 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:49.541563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:39073)
0809 04:45:49.541885 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:49.541898 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:49.541992 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:49.542307 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:49.542310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:49.542519 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:49.542630 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:49.542636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.543476 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.543480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:49.544277 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:49.544288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.544783 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.544787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:49.544863 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:49.545442 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:49.545458 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:49.547870 (+ 2412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":110,"threads_started":1}
W20260809 04:45:49.548368 18350 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:49.541699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:39073 (local address 127.7.171.62:45137)
0809 04:45:49.541961 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:49.541965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:49.541979 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:49.542097 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:49.542099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:49.542145 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:49.542218 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:49.542763 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.543328 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.544398 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.544626 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.544703 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:49.548213 (+ 3510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:39073: BlockingRecv error: recv got EOF from 127.7.171.1:39073 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":101,"threads_started":1}
Aug 09 04:45:49 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:49 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:49.742195 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:49.736567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:54437)
0809 04:45:49.736865 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:49.736879 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:49.736973 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:49.737253 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:49.737255 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:49.737440 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:49.737542 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:49.737547 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.738467 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.738469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:49.739040 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:49.739047 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.739465 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.739467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:49.739526 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:49.740152 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:49.740164 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:49.742069 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":93,"threads_started":1}
W20260809 04:45:49.742385 18350 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:49.736593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:54437 (local address 127.7.171.62:45137)
0809 04:45:49.736783 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:49.736788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:49.736934 (+ 146us) server_negotiation.cc:407] Connection header received
0809 04:45:49.737062 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:49.737064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:49.737138 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:49.737211 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:49.737669 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.738366 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.739149 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.739365 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.739443 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:49.742245 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:54437: BlockingRecv error: recv got EOF from 127.7.171.2:54437 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 04:45:49 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:49 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:49.955133 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:49.949380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:35521)
0809 04:45:49.949679 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:49.949692 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:49.949793 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:49.950038 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:49.950040 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:49.950249 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:49.950401 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:49.950411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.951372 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.951375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:49.952109 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:49.952120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.952557 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.952559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:49.952628 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:49.953187 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:49.953199 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:49.954993 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":123,"threads_started":1}
W20260809 04:45:49.955250 18350 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:49.949443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:35521 (local address 127.7.171.62:45137)
0809 04:45:49.949596 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:49.949601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:49.949771 (+ 170us) server_negotiation.cc:407] Connection header received
0809 04:45:49.949874 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:49.949876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:49.949928 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:49.950009 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:49.950531 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.951254 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.952221 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.952462 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.952542 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:49.955138 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:35521: BlockingRecv error: recv got EOF from 127.7.171.3:35521 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 04:45:50 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:50 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:50.554666 18354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:50.548968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:41647)
0809 04:45:50.549312 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:50.549327 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:50.549426 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:50.549708 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:50.549710 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:50.549897 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:50.550002 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:50.550007 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.550808 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.550811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:50.551389 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:50.551397 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.551835 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.551838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:50.551900 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:50.552469 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:50.552481 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:50.554536 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":105,"threads_started":1}
W20260809 04:45:50.555112 18355 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:50.549102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:41647 (local address 127.7.171.62:45137)
0809 04:45:50.549382 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:50.549387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:50.549402 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:50.549533 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:50.549535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:50.549585 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:50.549674 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:50.550112 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.550702 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.551495 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.551738 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.551819 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:50.554883 (+ 3064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:41647: BlockingRecv error: recv got EOF from 127.7.171.1:41647 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":66,"threads_started":1}
Aug 09 04:45:50 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:50 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:50.749321 18356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:50.743126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:41749)
0809 04:45:50.743421 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:50.743436 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:50.743540 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:50.743815 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:50.743818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:50.744119 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:50.744226 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:50.744233 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.745136 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.745138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:50.745698 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:50.745706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.746230 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.746233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:50.746303 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:50.746902 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:50.746921 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:50.749197 (+ 2276us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":104,"threads_started":1}
W20260809 04:45:50.749516 18355 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:50.743269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:41749 (local address 127.7.171.62:45137)
0809 04:45:50.743427 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:50.743432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:50.743491 (+ 59us) server_negotiation.cc:407] Connection header received
0809 04:45:50.743628 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:50.743630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:50.743675 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:50.743753 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:50.744351 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.745017 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.745808 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.746095 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.746182 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:50.749345 (+ 3163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:41749: BlockingRecv error: recv got EOF from 127.7.171.2:41749 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 04:45:50 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:50 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:50.961710 18357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:50.956044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:33395)
0809 04:45:50.956362 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:50.956375 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:50.956476 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:50.956713 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:50.956715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:50.956905 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:50.957001 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:50.957006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.957754 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.957757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:50.958315 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:50.958323 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.958751 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.958754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:50.958819 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:50.959420 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:50.959433 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:50.961593 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":80,"threads_started":1}
W20260809 04:45:50.961877 18355 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:50.956175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:33395 (local address 127.7.171.62:45137)
0809 04:45:50.956355 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:50.956359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:50.956428 (+ 69us) server_negotiation.cc:407] Connection header received
0809 04:45:50.956555 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:50.956557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:50.956606 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:50.956666 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:50.957124 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.957647 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.958419 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:50.958646 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:50.958724 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:50.961721 (+ 2997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:33395: BlockingRecv error: recv got EOF from 127.7.171.3:33395 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 04:45:51 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:51 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:51.561295 18358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:51.555591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:33867)
0809 04:45:51.555922 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:51.555937 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:51.556024 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:51.556380 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:51.556382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:51.556574 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:51.556678 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:51.556684 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.557536 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.557538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:51.558107 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:51.558114 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.558560 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.558562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:51.558625 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:51.559209 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:51.559221 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:51.561176 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":97,"threads_started":1}
W20260809 04:45:51.561560 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:51.555649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:33867 (local address 127.7.171.62:45137)
0809 04:45:51.555980 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:51.555985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:51.556000 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:51.556190 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:51.556193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:51.556246 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:51.556341 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:51.556825 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.557427 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.558222 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.558452 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.558541 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:51.561369 (+ 2828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:33867: BlockingRecv error: recv got EOF from 127.7.171.1:33867 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":117,"threads_started":1}
Aug 09 04:45:51 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:51 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:51.756309 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:51.750296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:41087)
0809 04:45:51.750640 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:51.750655 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:51.750767 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:51.751023 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:51.751025 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:51.751225 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:51.751325 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:51.751330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.752300 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.752307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:51.753011 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:51.753022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.753556 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.753559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:51.753658 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:51.754175 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:51.754187 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:51.756158 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":138,"threads_started":1}
W20260809 04:45:51.756374 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:51.750389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:41087 (local address 127.7.171.62:45137)
0809 04:45:51.750567 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:51.750573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:51.750748 (+ 175us) server_negotiation.cc:407] Connection header received
0809 04:45:51.750856 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:51.750858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:51.750905 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:51.750978 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:51.751437 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.752174 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.753144 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.753439 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.753548 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:51.756277 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:41087: BlockingRecv error: recv got EOF from 127.7.171.2:41087 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 04:45:51 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:51 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:51.968732 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:51.962662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:51011)
0809 04:45:51.963002 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:51.963017 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:51.963139 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:51.963404 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:51.963407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:51.963646 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:51.963760 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:51.963766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.964734 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.964737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:51.965318 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:51.965327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.965998 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.966002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:51.966101 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:51.966685 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:51.966697 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:51.968596 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":154,"threads_started":1}
W20260809 04:45:51.968889 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:51.962817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:51011 (local address 127.7.171.62:45137)
0809 04:45:51.962969 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:51.962972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:51.963172 (+ 200us) server_negotiation.cc:407] Connection header received
0809 04:45:51.963231 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:51.963233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:51.963283 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:51.963364 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:51.963886 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.964613 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.965442 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.965701 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.965790 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:51.968736 (+ 2946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:51011: BlockingRecv error: recv got EOF from 127.7.171.3:51011 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 04:45:52 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:52 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:52.568346 18364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:52.562337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:50435)
0809 04:45:52.562682 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:52.562699 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:52.562817 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:52.563188 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:52.563192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:52.563459 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:52.563597 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:52.563606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.564598 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.564601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:52.565164 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:52.565172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.565595 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.565598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:52.565664 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:52.566183 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:52.566195 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:52.568228 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":118,"threads_started":1}
W20260809 04:45:52.568655 18365 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:52.562432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:50435 (local address 127.7.171.62:45137)
0809 04:45:52.562786 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:52.562791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:52.562805 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:45:52.562959 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:52.562961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:52.563014 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:52.563133 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:52.563758 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.564473 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.565269 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.565488 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.565562 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:52.568477 (+ 2915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:50435: BlockingRecv error: recv got EOF from 127.7.171.1:50435 (error 108)
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":100,"threads_started":1}
Aug 09 04:45:52 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:52 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:52.762905 18366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:52.757257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:50061)
0809 04:45:52.757565 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:52.757580 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:52.757682 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:52.757934 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:52.757937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:52.758143 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:52.758244 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:52.758249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.758989 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.758991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:52.759560 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:52.759567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.759969 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.759971 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:52.760054 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:52.760600 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:52.760611 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:52.762721 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":89,"threads_started":1}
W20260809 04:45:52.762969 18365 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:52.757345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:50061 (local address 127.7.171.62:45137)
0809 04:45:52.757495 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:52.757499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:52.757649 (+ 150us) server_negotiation.cc:407] Connection header received
0809 04:45:52.757769 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:52.757771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:52.757815 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:52.757891 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:52.758350 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.758891 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.759660 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.759871 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.759965 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:52.762856 (+ 2891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:50061: BlockingRecv error: recv got EOF from 127.7.171.2:50061 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 04:45:52 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:52 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:52.975364 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:52.969534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:47113)
0809 04:45:52.969840 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:52.969855 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:52.969948 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:52.970216 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:52.970219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:52.970402 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:52.970520 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:52.970525 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.971251 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.971254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:52.971844 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:52.971853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.972361 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.972365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:52.972464 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:52.972947 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:52.972963 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:52.975187 (+ 2224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":134,"threads_started":1}
W20260809 04:45:52.975488 18365 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:52.969760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:47113 (local address 127.7.171.62:45137)
0809 04:45:52.969903 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:52.969907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:52.969920 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:45:52.970047 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:52.970049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:52.970106 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:52.970179 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:52.970620 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.971146 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.971946 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.972220 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.972328 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:52.975333 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:47113: BlockingRecv error: recv got EOF from 127.7.171.3:47113 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 04:45:53 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:53 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:53.574882 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.569253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:56379)
0809 04:45:53.569550 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:53.569565 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:53.569659 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:53.569912 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:53.569915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:53.570121 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:53.570221 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.570226 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.570931 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.570933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.571507 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.571515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.571914 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.571917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.571978 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.572512 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:53.572524 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:53.574701 (+ 2177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":75,"threads_started":1}
W20260809 04:45:53.575098 18370 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:53.569321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:56379 (local address 127.7.171.62:45137)
0809 04:45:53.569610 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.569614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.569627 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:45:53.569746 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.569748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.569794 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.569875 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:53.570318 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.570840 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.571603 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.571819 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.571892 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.574888 (+ 2996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:56379: BlockingRecv error: recv got EOF from 127.7.171.1:56379 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":104,"threads_started":1}
Aug 09 04:45:53 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:53 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:53.769369 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.763820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:42491)
0809 04:45:53.764149 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:53.764166 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:53.764286 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:53.764522 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:53.764524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:53.764707 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:53.764808 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.764813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.765540 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.765542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.766134 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.766142 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.766562 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.766564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.766624 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.767200 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:53.767212 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:53.769195 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":70,"threads_started":1}
W20260809 04:45:53.769436 18370 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:53.763954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:42491 (local address 127.7.171.62:45137)
0809 04:45:53.764181 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.764186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.764250 (+ 64us) server_negotiation.cc:407] Connection header received
0809 04:45:53.764371 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.764373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.764419 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.764485 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:53.764905 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.765443 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.766244 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.766462 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.766542 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.769327 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:42491: BlockingRecv error: recv got EOF from 127.7.171.2:42491 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 04:45:53 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:53 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:53.986382 18372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.976151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:35681)
0809 04:45:53.976468 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:53.976482 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:53.976591 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:53.976868 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:53.976871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:53.977072 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:53.977174 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.977180 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.977885 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.977888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.978440 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.978447 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.978859 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.978862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.978923 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.984348 (+ 5425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:53.984362 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:53.986253 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":135,"threads_started":1}
W20260809 04:45:53.986577 18370 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:53.976189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:35681 (local address 127.7.171.62:45137)
0809 04:45:53.976345 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.976350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.976539 (+ 189us) server_negotiation.cc:407] Connection header received
0809 04:45:53.976665 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.976668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.976729 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.976813 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:53.977277 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.977791 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.978535 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.978760 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.978836 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.986443 (+ 7607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:35681: BlockingRecv error: recv got EOF from 127.7.171.3:35681 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 04:45:54 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:54 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:54.581348 18373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:54.575779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:47215)
0809 04:45:54.576090 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:54.576105 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:54.576196 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:54.576450 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:54.576452 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:54.576646 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:54.576746 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:54.576751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.577466 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.577468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:54.578038 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:54.578046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.578467 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.578469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:54.578532 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:54.579135 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:54.579147 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:54.581143 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":94,"threads_started":1}
W20260809 04:45:54.581410 18375 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:54.575836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:47215 (local address 127.7.171.62:45137)
0809 04:45:54.576136 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:54.576141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:54.576163 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:45:54.576290 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:54.576292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:54.576339 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:54.576422 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:54.576851 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.577373 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.578144 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.578367 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.578442 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:54.581277 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:47215: BlockingRecv error: recv got EOF from 127.7.171.1:47215 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":65,"threads_started":1}
Aug 09 04:45:54 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:54 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:54.776239 18376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:54.770313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:46005)
0809 04:45:54.770689 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:54.770708 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:54.770835 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:54.771157 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:54.771163 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:54.771376 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:54.771486 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:54.771492 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.772581 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.772584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:54.773182 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:54.773191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.773594 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.773597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:54.773670 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:54.774225 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:54.774237 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:54.776041 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":126,"threads_started":1}
W20260809 04:45:54.776398 18375 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:54.770474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:46005 (local address 127.7.171.62:45137)
0809 04:45:54.770624 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:54.770628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:54.770806 (+ 178us) server_negotiation.cc:407] Connection header received
0809 04:45:54.770929 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:54.770932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:54.770989 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:54.771111 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:54.771639 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.772469 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.773287 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.773501 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.773575 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:54.776244 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:46005: BlockingRecv error: recv got EOF from 127.7.171.2:46005 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 04:45:54 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:54 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:54.993152 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:54.987251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:35819)
0809 04:45:54.987559 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:54.987576 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:54.987733 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:54.988007 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:54.988010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:54.988259 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:54.988355 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:54.988362 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.989149 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.989154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:54.989728 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:54.989736 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.990224 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.990227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:54.990316 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:54.990914 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:54.990927 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:54.993013 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":102,"threads_started":1}
W20260809 04:45:54.993314 18375 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:54.987386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:35819 (local address 127.7.171.62:45137)
0809 04:45:54.987559 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:54.987566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:54.987691 (+ 125us) server_negotiation.cc:407] Connection header received
0809 04:45:54.987835 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:54.987838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:54.987895 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:54.987984 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:54.988484 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.989026 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.989853 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.990104 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.990218 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:54.993159 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:35819: BlockingRecv error: recv got EOF from 127.7.171.3:35819 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 04:45:55 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:55 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:55.588410 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:55.582307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:57775)
0809 04:45:55.582653 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:55.582668 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:55.582770 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:55.583038 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:55.583040 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:55.583243 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:55.583345 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:55.583350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.584177 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.584181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:55.584979 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:55.584990 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.585527 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.585530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:55.585619 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:55.586163 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:55.586175 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:55.588289 (+ 2114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":101,"threads_started":1}
W20260809 04:45:55.588588 18379 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:55.582467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:57775 (local address 127.7.171.62:45137)
0809 04:45:55.582810 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:55.582814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:55.582829 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:45:55.582873 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:55.582876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:55.582919 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:55.583002 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:55.583462 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.584005 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.585081 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.585405 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.585489 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:55.588428 (+ 2939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:57775: BlockingRecv error: recv got EOF from 127.7.171.1:57775 (error 108)
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":104,"threads_started":1}
Aug 09 04:45:55 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:55 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:55.782742 18380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:55.777047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:57327)
0809 04:45:55.777394 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:55.777406 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:55.777546 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:55.777831 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:55.777835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:55.778056 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:55.778176 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:55.778181 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.778963 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.778968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:55.779565 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:55.779574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.780042 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.780046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:55.780167 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:55.780745 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:55.780758 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:55.782569 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":95,"threads_started":1}
W20260809 04:45:55.782846 18379 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:55.777153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:57327 (local address 127.7.171.62:45137)
0809 04:45:55.777330 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:55.777336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:55.777484 (+ 148us) server_negotiation.cc:407] Connection header received
0809 04:45:55.777656 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:55.777660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:55.777715 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:55.777803 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:55.778299 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.778849 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.779691 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.779925 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.780038 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:55.782726 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:57327: BlockingRecv error: recv got EOF from 127.7.171.2:57327 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 04:45:55 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:55 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.3@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:56.000020 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:55.994112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.3:45075)
0809 04:45:55.994500 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:55.994518 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:55.994675 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:55.995074 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:55.995077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:55.995286 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:55.995380 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:55.995386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.996207 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.996210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:55.996777 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:55.996785 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.997328 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.997332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:55.997420 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:55.997971 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:55.998005 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:55.999891 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":113,"threads_started":1}
W20260809 04:45:56.000353 18379 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:55.994329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:45075 (local address 127.7.171.62:45137)
0809 04:45:55.994506 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:55.994514 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:55.994611 (+ 97us) server_negotiation.cc:407] Connection header received
0809 04:45:55.994788 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:55.994791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:55.994851 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:55.994938 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:55.995505 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.996037 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.996898 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:55.997203 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:55.997321 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:56.000211 (+ 2890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.3:45075: BlockingRecv error: recv got EOF from 127.7.171.3:45075 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 04:45:56 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:56 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.1@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:56.595008 18382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:56.589376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.1:45457)
0809 04:45:56.589672 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:56.589686 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:56.589770 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:56.590004 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:56.590007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:56.590224 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:56.590321 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:56.590326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:56.591066 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:56.591069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:56.591643 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:56.591651 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:56.592137 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:56.592142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:56.592246 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:56.592782 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:56.592794 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:56.594833 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":75,"threads_started":1}
W20260809 04:45:56.595090 18383 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:56.589437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:45457 (local address 127.7.171.62:45137)
0809 04:45:56.589739 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:56.589742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:56.589754 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:45:56.589852 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:56.589854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:56.589896 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:56.589964 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:56.590424 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:56.590960 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:56.591746 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:56.591967 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:56.592113 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:56.594964 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.1:45457: BlockingRecv error: recv got EOF from 127.7.171.1:45457 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":95,"threads_started":1}
Aug 09 04:45:56 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:45:56 dist-test-slave-1zqn krb5kdc[17176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.171.2@KRBTEST.COM for oryx/127.7.171.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:45:56.789080 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:56.783742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45137 (local address 127.7.171.2:56065)
0809 04:45:56.784096 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:56.784115 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:56.784227 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:56.784466 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:56.784468 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:56.784671 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:56.784768 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:56.784773 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:56.785476 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:56.785478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:56.786044 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:56.786051 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:56.786442 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:56.786444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:56.786504 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:56.787010 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:56.787021 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:56.788959 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.171.62:45137: Server oryx/127.7.171.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":108,"threads_started":1}
W20260809 04:45:56.789232 18383 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:56.783855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:56065 (local address 127.7.171.62:45137)
0809 04:45:56.784010 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:56.784014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:56.784196 (+ 182us) server_negotiation.cc:407] Connection header received
0809 04:45:56.784307 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:56.784309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:56.784353 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:56.784426 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:56.784865 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:56.785382 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:56.786139 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:56.786351 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:56.786423 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:56.789105 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.171.2:56065: BlockingRecv error: recv got EOF from 127.7.171.2:56065 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:45:56.886718 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 17845
I20260809 04:45:56.896209 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 18014
I20260809 04:45:56.904372 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 18164
I20260809 04:45:56.912531 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 17769
2026-08-09T04:45:56Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18188 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T04:45:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:56Z Disabled control of system clock
I20260809 04:45:56.946771 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:38079
--webserver_interface=127.7.171.62
--webserver_port=0
--builtin_ntp_servers=127.7.171.20:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:38079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 04:45:57.062625 18388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:57.062879 18388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:57.062911 18388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:57.066942 18388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:57.067003 18388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:57.067018 18388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:57.067034 18388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:57.067050 18388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:57.071874 18388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:38079
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:38079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:57.073024 18388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:57.074409 18388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:57.083639 18397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:57.083634 18395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:57.083628 18394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:57.084210 18388 server_base.cc:1034] running on GCE node
I20260809 04:45:57.084793 18388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:57.085992 18388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:57.087200 18388 hybrid_clock.cc:648] HybridClock initialized: now 1786250757087180 us; error 40 us; skew 500 ppm
I20260809 04:45:57.089318 18388 webserver.cc:460] Webserver started at http://127.7.171.62:45021/ using document root <none> and password file <none>
I20260809 04:45:57.089941 18388 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:57.090039 18388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:57.090276 18388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:57.092427 18388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "e041087adf39492ab56ab4c1427b3d39"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "3faec501ebd031e3a64e2da85f5031c3"
server_key_iv: "3d1ae439ea9b4fb9178d3551bc16491f"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.093022 18388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "e041087adf39492ab56ab4c1427b3d39"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "3faec501ebd031e3a64e2da85f5031c3"
server_key_iv: "3d1ae439ea9b4fb9178d3551bc16491f"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.096971 18388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:45:57.099615 18403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:57.101099 18388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:57.101274 18388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "e041087adf39492ab56ab4c1427b3d39"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "3faec501ebd031e3a64e2da85f5031c3"
server_key_iv: "3d1ae439ea9b4fb9178d3551bc16491f"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.101459 18388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:57.126595 18388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:57.127522 18388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:57.127753 18388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:57.137457 18388 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:38079
I20260809 04:45:57.137470 18464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:38079 every 8 connection(s)
I20260809 04:45:57.138821 18388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:57.141831 18465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:57.143493 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 18388
I20260809 04:45:57.143623 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:57.143934 7852 external_mini_cluster.cc:1428] Setting key 1584ef2bc1fa1bc98c640782757a1be9
I20260809 04:45:57.149477 18465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39: Bootstrap starting.
I20260809 04:45:57.152019 18465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:57.152874 18465 log.cc:824] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:57.155241 18465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39: No bootstrap required, opened a new log
I20260809 04:45:57.155651 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:57.145481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45850 (local address 127.7.171.62:38079)
0809 04:45:57.146365 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:57.146390 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:57.146441 (+ 51us) server_negotiation.cc:407] Connection header received
0809 04:45:57.147626 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:57.147653 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:57.148269 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:57.148707 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:57.149217 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.150898 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.152321 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.152795 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.152913 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:57.153818 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:57.153823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:57.153836 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:57.154293 (+ 457us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:57.154348 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:57.154369 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:57.154481 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:57.154621 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 04:45:57.154878 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":142,"threads_started":1}
I20260809 04:45:57.159574 18465 raft_consensus.cc:348] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e041087adf39492ab56ab4c1427b3d39" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38079 } }
I20260809 04:45:57.159803 18465 raft_consensus.cc:374] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:57.159863 18465 raft_consensus.cc:729] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e041087adf39492ab56ab4c1427b3d39, State: Initialized, Role: FOLLOWER
I20260809 04:45:57.160323 18465 consensus_queue.cc:260] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e041087adf39492ab56ab4c1427b3d39" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38079 } }
I20260809 04:45:57.160495 18465 raft_consensus.cc:388] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:57.160566 18465 raft_consensus.cc:482] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:57.160671 18465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:57.161787 18465 raft_consensus.cc:504] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e041087adf39492ab56ab4c1427b3d39" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38079 } }
I20260809 04:45:57.162133 18465 leader_election.cc:302] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e041087adf39492ab56ab4c1427b3d39; no voters:
I20260809 04:45:57.162454 18465 leader_election.cc:288] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:57.162695 18471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:57.162957 18471 raft_consensus.cc:686] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [term 1 LEADER]: Becoming Leader. State: Replica: e041087adf39492ab56ab4c1427b3d39, State: Running, Role: LEADER
I20260809 04:45:57.163314 18471 consensus_queue.cc:237] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e041087adf39492ab56ab4c1427b3d39" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38079 } }
I20260809 04:45:57.163806 18465 sys_catalog.cc:564] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:57.165148 18473 sys_catalog.cc:455] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e041087adf39492ab56ab4c1427b3d39. Latest consensus state: current_term: 1 leader_uuid: "e041087adf39492ab56ab4c1427b3d39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e041087adf39492ab56ab4c1427b3d39" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38079 } } }
I20260809 04:45:57.165201 18472 sys_catalog.cc:455] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e041087adf39492ab56ab4c1427b3d39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e041087adf39492ab56ab4c1427b3d39" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38079 } } }
I20260809 04:45:57.165282 18473 sys_catalog.cc:458] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:57.165300 18472 sys_catalog.cc:458] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:57.165695 18481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:57.168596 18481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:57.174168 18481 catalog_manager.cc:1347] Generated new cluster ID: 1f2ae364169240879b29ed4a8b71da77
I20260809 04:45:57.174247 18481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:57.191210 18481 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:57.192222 18481 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:57.198560 18481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e041087adf39492ab56ab4c1427b3d39: Generated new TSK 0
I20260809 04:45:57.199337 18481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:45:57.208647 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:38079
--builtin_ntp_servers=127.7.171.20:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 04:45:57.323777 18494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:57.324103 18494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:57.324186 18494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:57.328236 18494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:57.328315 18494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:57.328409 18494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:57.333097 18494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:38079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:57.334158 18494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:57.335564 18494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:45:57.345512 18494 server_base.cc:1034] running on GCE node
W20260809 04:45:57.345451 18499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:57.345451 18502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:57.345481 18500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:57.346150 18494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:57.346793 18494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:57.347983 18494 hybrid_clock.cc:648] HybridClock initialized: now 1786250757347965 us; error 39 us; skew 500 ppm
I20260809 04:45:57.349992 18494 webserver.cc:460] Webserver started at http://127.7.171.1:34361/ using document root <none> and password file <none>
I20260809 04:45:57.350605 18494 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:57.350703 18494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:57.350940 18494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:57.353147 18494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "88f0d793aa3148999377b1481afa8bf9"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "c322ebb8442adf44867e7c1be20852ab"
server_key_iv: "76c35c16dee4be175a5284fb49cf4ce3"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.353744 18494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "88f0d793aa3148999377b1481afa8bf9"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "c322ebb8442adf44867e7c1be20852ab"
server_key_iv: "76c35c16dee4be175a5284fb49cf4ce3"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.357765 18494 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:45:57.360368 18509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:57.361609 18494 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:45:57.361752 18494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "88f0d793aa3148999377b1481afa8bf9"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "c322ebb8442adf44867e7c1be20852ab"
server_key_iv: "76c35c16dee4be175a5284fb49cf4ce3"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.361866 18494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:57.389633 18494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:57.390398 18494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:57.390625 18494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:57.391264 18494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:57.392273 18494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:57.392350 18494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:57.392424 18494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:57.392469 18494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:57.401672 18494 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:37935
I20260809 04:45:57.401695 18629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:37935 every 8 connection(s)
I20260809 04:45:57.402675 18494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:57.406932 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 18494
I20260809 04:45:57.407110 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:57.407486 7852 external_mini_cluster.cc:1428] Setting key e908c1926e00f56eac545631c8227881
I20260809 04:45:57.409726 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:38079
--builtin_ntp_servers=127.7.171.20:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 04:45:57.413802 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:57.404717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:50081 (local address 127.7.171.62:38079)
0809 04:45:57.404871 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:57.404875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:57.406001 (+ 1126us) server_negotiation.cc:407] Connection header received
0809 04:45:57.407791 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:57.407793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:57.407846 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:57.407954 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:57.409072 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.409626 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.411395 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.411685 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.411816 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:57.413274 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:57.413280 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:57.413288 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:57.413300 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:57.413319 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:57.413327 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:57.413407 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:57.413588 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 04:45:57.413660 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:45:57.414690 18633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:57.405167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:38079 (local address 127.7.171.1:50081)
0809 04:45:57.405843 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:57.405885 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:57.407535 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:57.408175 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:57.408182 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:57.408288 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:57.408931 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:57.408947 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.409768 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.409772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:57.411238 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:57.411253 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.411852 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.411857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:57.412254 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:57.412957 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:57.412959 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:57.413000 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:57.413107 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:57.413422 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:57.413432 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:57.413438 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:45:57.413582 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 04:45:57.413870 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":124,"threads_started":1}
I20260809 04:45:57.416025 18630 heartbeater.cc:347] Connected to a master server at 127.7.171.62:38079
I20260809 04:45:57.416334 18630 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:57.417029 18630 heartbeater.cc:511] Master 127.7.171.62:38079 requested a full tablet report, sending...
I20260809 04:45:57.418740 18421 ts_manager.cc:194] Registered new tserver with Master: 88f0d793aa3148999377b1481afa8bf9 (127.7.171.1:37935)
I20260809 04:45:57.419919 18421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.1:50081
W20260809 04:45:57.525475 18634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:57.525753 18634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:57.525789 18634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:57.529912 18634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:57.529963 18634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:57.530056 18634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:57.534711 18634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:38079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:57.535790 18634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:57.537326 18634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:57.547227 18641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:57.547231 18643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:57.547233 18640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:57.547453 18634 server_base.cc:1034] running on GCE node
I20260809 04:45:57.547938 18634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:57.548619 18634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:57.549813 18634 hybrid_clock.cc:648] HybridClock initialized: now 1786250757549776 us; error 48 us; skew 500 ppm
I20260809 04:45:57.551614 18634 webserver.cc:460] Webserver started at http://127.7.171.2:42857/ using document root <none> and password file <none>
I20260809 04:45:57.552271 18634 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:57.552366 18634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:57.552618 18634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:57.554764 18634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "6b4c7ad28ddd45ae92b6d93cc1efa05d"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "705a9db52f6345655c7c8e8e72b8c6d3"
server_key_iv: "8018e9aad4cca574f9b929642c800f9c"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.555348 18634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "6b4c7ad28ddd45ae92b6d93cc1efa05d"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "705a9db52f6345655c7c8e8e72b8c6d3"
server_key_iv: "8018e9aad4cca574f9b929642c800f9c"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.559250 18634 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:45:57.561865 18650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:57.563059 18634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:45:57.563203 18634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "6b4c7ad28ddd45ae92b6d93cc1efa05d"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "705a9db52f6345655c7c8e8e72b8c6d3"
server_key_iv: "8018e9aad4cca574f9b929642c800f9c"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.563324 18634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:57.590523 18634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:57.591315 18634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:57.591531 18634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:57.592199 18634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:57.593202 18634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:57.593278 18634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:57.593353 18634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:57.593396 18634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:57.603884 18634 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:33753
I20260809 04:45:57.603916 18773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:33753 every 8 connection(s)
I20260809 04:45:57.604880 18634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:57.607892 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 18634
I20260809 04:45:57.608011 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:57.608387 7852 external_mini_cluster.cc:1428] Setting key 5a70b79f05496f4f7656a4a45892ecf9
I20260809 04:45:57.610752 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:38079
--builtin_ntp_servers=127.7.171.20:38973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 04:45:57.615465 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:57.606733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:47909 (local address 127.7.171.62:38079)
0809 04:45:57.606881 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:57.606885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:57.608219 (+ 1334us) server_negotiation.cc:407] Connection header received
0809 04:45:57.609969 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:57.609972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:57.610028 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:57.610115 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:57.611384 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.611904 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.613457 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.613685 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.613804 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:57.614946 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:57.614954 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:57.614956 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:57.614968 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:57.614987 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:57.614995 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:57.615085 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:57.615246 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 04:45:57.615317 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:45:57.616434 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:57.607202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:38079 (local address 127.7.171.2:47909)
0809 04:45:57.608010 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:57.608099 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:57.609756 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:57.610335 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:57.610344 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:57.610458 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:57.611213 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:57.611235 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.612035 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.612042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:57.613273 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:57.613283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.613852 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.613856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:57.614023 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:57.614673 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:57.614675 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:57.614715 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:57.614783 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:57.615079 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:57.615090 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:57.615098 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:45:57.615245 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 04:45:57.615555 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":120,"threads_started":1}
I20260809 04:45:57.617652 18774 heartbeater.cc:347] Connected to a master server at 127.7.171.62:38079
I20260809 04:45:57.617944 18774 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:57.618692 18774 heartbeater.cc:511] Master 127.7.171.62:38079 requested a full tablet report, sending...
I20260809 04:45:57.620189 18421 ts_manager.cc:194] Registered new tserver with Master: 6b4c7ad28ddd45ae92b6d93cc1efa05d (127.7.171.2:33753)
I20260809 04:45:57.621084 18421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.2:47909
W20260809 04:45:57.727597 18778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:57.727896 18778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:57.727970 18778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:57.731972 18778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:57.732055 18778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:57.732177 18778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:57.736851 18778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:38079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:57.737920 18778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:57.739328 18778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:57.748998 18784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:57.748967 18787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:57.748960 18785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:57.749305 18778 server_base.cc:1034] running on GCE node
I20260809 04:45:57.749863 18778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:57.750545 18778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:57.751726 18778 hybrid_clock.cc:648] HybridClock initialized: now 1786250757751696 us; error 42 us; skew 500 ppm
I20260809 04:45:57.753679 18778 webserver.cc:460] Webserver started at http://127.7.171.3:36727/ using document root <none> and password file <none>
I20260809 04:45:57.754278 18778 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:57.754343 18778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:57.754601 18778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:57.756878 18778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "20301b929db64479bfb8affb5367f852"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "7a8e3464410f96722f4daa2337e60276"
server_key_iv: "a08a3da7a1d9e5e76d031e6324d41778"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.757525 18778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "20301b929db64479bfb8affb5367f852"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "7a8e3464410f96722f4daa2337e60276"
server_key_iv: "a08a3da7a1d9e5e76d031e6324d41778"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.761318 18778 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:57.763794 18793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:57.765069 18778 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:45:57.765297 18778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "20301b929db64479bfb8affb5367f852"
format_stamp: "Formatted at 2026-08-09 04:45:57 on dist-test-slave-1zqn"
server_key: "7a8e3464410f96722f4daa2337e60276"
server_key_iv: "a08a3da7a1d9e5e76d031e6324d41778"
server_key_version: "encryptionkey@0"
I20260809 04:45:57.765398 18778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:57.784653 18778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:57.785405 18778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:57.785617 18778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:57.786238 18778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:57.787210 18778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:57.787257 18778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:57.787321 18778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:57.787355 18778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:57.796944 18778 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:45043
I20260809 04:45:57.796988 18918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:45043 every 8 connection(s)
I20260809 04:45:57.797894 18778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:57.798604 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 18778
I20260809 04:45:57.798759 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:57.799130 7852 external_mini_cluster.cc:1428] Setting key 50a41e4e6b25bc58056780091dcc285c
I20260809 04:45:57.807642 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:57.799957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:39909 (local address 127.7.171.62:38079)
0809 04:45:57.800138 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:57.800145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:57.801445 (+ 1300us) server_negotiation.cc:407] Connection header received
0809 04:45:57.802954 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:57.802956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:57.803005 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:57.803080 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:57.804000 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.804613 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.805714 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.805923 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.805996 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:57.807120 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:57.807123 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:57.807125 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:57.807137 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:57.807156 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:57.807185 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:57.807267 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:57.807456 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 04:45:57.807517 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:45:57.808620 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:57.800482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:38079 (local address 127.7.171.3:39909)
0809 04:45:57.801297 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:57.801351 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:57.802659 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:57.803273 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:57.803280 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:57.803368 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:57.803873 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:57.803886 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.804724 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.804727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:57.805609 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:57.805617 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:57.806120 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:57.806125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:57.806294 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:57.806859 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:57.806861 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:57.806906 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:57.806973 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:57.807272 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:57.807281 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:57.807287 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:45:57.807389 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 04:45:57.807630 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":151,"threads_started":1}
I20260809 04:45:57.809768 18919 heartbeater.cc:347] Connected to a master server at 127.7.171.62:38079
I20260809 04:45:57.810024 18919 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:57.810556 18919 heartbeater.cc:511] Master 127.7.171.62:38079 requested a full tablet report, sending...
I20260809 04:45:57.811641 18421 ts_manager.cc:194] Registered new tserver with Master: 20301b929db64479bfb8affb5367f852 (127.7.171.3:45043)
I20260809 04:45:57.812410 18421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.3:39909
I20260809 04:45:57.813341 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:45:57.816352 18468 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:57.815189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45852 (local address 127.7.171.62:38079)
0809 04:45:57.815340 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:57.815345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:57.815561 (+ 216us) server_negotiation.cc:407] Connection header received
0809 04:45:57.815724 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:57.815727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:57.815815 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:57.815893 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:57.816237 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45852: BlockingRecv error: recv got EOF from 127.0.0.1:45852 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:45:57.816819 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 18494
I20260809 04:45:57.828445 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 18634
I20260809 04:45:57.837116 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 18778
I20260809 04:45:57.845378 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 18388
2026-08-09T04:45:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (933 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T04:45:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:57Z Disabled control of system clock
I20260809 04:45:57.879473 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43225
--webserver_interface=127.7.171.62
--webserver_port=0
--builtin_ntp_servers=127.7.171.20:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:43225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 04:45:57.992187 18930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:57.992451 18930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:57.992489 18930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:57.996511 18930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:57.996569 18930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:57.996584 18930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:57.996603 18930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:57.996618 18930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:58.001313 18930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:43225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:58.002372 18930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:58.003762 18930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:58.012450 18940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:58.012440 18938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:58.012444 18936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:58.012812 18930 server_base.cc:1034] running on GCE node
I20260809 04:45:58.013374 18930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:58.014479 18930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:58.015650 18930 hybrid_clock.cc:648] HybridClock initialized: now 1786250758015632 us; error 32 us; skew 500 ppm
I20260809 04:45:58.017557 18930 webserver.cc:460] Webserver started at http://127.7.171.62:38977/ using document root <none> and password file <none>
I20260809 04:45:58.018148 18930 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:58.018210 18930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:58.018478 18930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:58.020666 18930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "679b3928f95b41aa88d696b8a511f8ac"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "4bb8f27e011dedb0fbecac0b732a8b5b"
server_key_iv: "cb0ccabdda7a657041d62e3e13bde83f"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.021307 18930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "679b3928f95b41aa88d696b8a511f8ac"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "4bb8f27e011dedb0fbecac0b732a8b5b"
server_key_iv: "cb0ccabdda7a657041d62e3e13bde83f"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.025076 18930 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:45:58.027472 18946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:58.028748 18930 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:58.028891 18930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "679b3928f95b41aa88d696b8a511f8ac"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "4bb8f27e011dedb0fbecac0b732a8b5b"
server_key_iv: "cb0ccabdda7a657041d62e3e13bde83f"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.029008 18930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:58.033663 18930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:58.034312 18930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:58.034497 18930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:58.041383 18930 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:43225
I20260809 04:45:58.041394 19005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:43225 every 8 connection(s)
I20260809 04:45:58.042322 18930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:58.045470 19006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:58.045616 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 18930
I20260809 04:45:58.045734 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:45:58.046036 7852 external_mini_cluster.cc:1428] Setting key 6192d8542b37c79ad1c686215900a171
I20260809 04:45:58.051529 19009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:58.047430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.7.171.62:43225)
0809 04:45:58.048116 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:58.048137 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:58.048173 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:45:58.048879 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:58.048896 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:58.049239 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:58.049575 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:58.050343 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:58.050348 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:58.050360 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:58.050370 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:58.050419 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:58.050435 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:58.050584 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:58.050663 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 04:45:58.050888 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":134,"threads_started":1}
I20260809 04:45:58.052425 19006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac: Bootstrap starting.
I20260809 04:45:58.054724 19006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:58.055409 19006 log.cc:824] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:58.057287 19006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac: No bootstrap required, opened a new log
I20260809 04:45:58.061110 19006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679b3928f95b41aa88d696b8a511f8ac" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43225 } }
I20260809 04:45:58.061326 19006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:58.061368 19006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679b3928f95b41aa88d696b8a511f8ac, State: Initialized, Role: FOLLOWER
I20260809 04:45:58.061779 19006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679b3928f95b41aa88d696b8a511f8ac" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43225 } }
I20260809 04:45:58.061898 19006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:58.061965 19006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:58.062067 19006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:58.062937 19006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679b3928f95b41aa88d696b8a511f8ac" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43225 } }
I20260809 04:45:58.063292 19006 leader_election.cc:302] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 679b3928f95b41aa88d696b8a511f8ac; no voters:
I20260809 04:45:58.063642 19006 leader_election.cc:288] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:58.063763 19011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:58.064026 19011 raft_consensus.cc:686] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [term 1 LEADER]: Becoming Leader. State: Replica: 679b3928f95b41aa88d696b8a511f8ac, State: Running, Role: LEADER
I20260809 04:45:58.064410 19011 consensus_queue.cc:237] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679b3928f95b41aa88d696b8a511f8ac" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43225 } }
I20260809 04:45:58.064894 19006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:58.066030 19012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "679b3928f95b41aa88d696b8a511f8ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679b3928f95b41aa88d696b8a511f8ac" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43225 } } }
I20260809 04:45:58.066174 19012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:58.066486 19013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 679b3928f95b41aa88d696b8a511f8ac. Latest consensus state: current_term: 1 leader_uuid: "679b3928f95b41aa88d696b8a511f8ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679b3928f95b41aa88d696b8a511f8ac" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43225 } } }
I20260809 04:45:58.066581 19022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:58.066675 19013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:58.069444 19022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:58.075397 19022 catalog_manager.cc:1347] Generated new cluster ID: 2aa77216f673404982026975ef3ec9d7
I20260809 04:45:58.075466 19022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:58.118155 19022 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:58.118549 19022 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:58.124954 19022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 679b3928f95b41aa88d696b8a511f8ac: Generated new TSK 0
I20260809 04:45:58.125667 19022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:45:58.131639 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43225
--builtin_ntp_servers=127.7.171.20:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 04:45:58.251354 19032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:58.251628 19032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:58.251701 19032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:58.255820 19032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:58.255880 19032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:58.255985 19032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:45:58.261080 19032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:58.262174 19032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:58.263608 19032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:58.273952 19037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:58.273916 19040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:58.273927 19038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:58.274010 19032 server_base.cc:1034] running on GCE node
I20260809 04:45:58.274528 19032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:58.275202 19032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:58.276407 19032 hybrid_clock.cc:648] HybridClock initialized: now 1786250758276393 us; error 30 us; skew 500 ppm
I20260809 04:45:58.278242 19032 webserver.cc:460] Webserver started at http://127.7.171.1:33535/ using document root <none> and password file <none>
I20260809 04:45:58.278867 19032 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:58.278965 19032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:58.279214 19032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:58.281428 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "ac67e9740bbb4f4f9b1fa4beead80ff5"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "199900be3858292e32416d66654ccc3a"
server_key_iv: "947bb13131066c5589be66c4cc3fb9ab"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.282049 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "ac67e9740bbb4f4f9b1fa4beead80ff5"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "199900be3858292e32416d66654ccc3a"
server_key_iv: "947bb13131066c5589be66c4cc3fb9ab"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.285995 19032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:45:58.288846 19048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:58.290103 19032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:58.290244 19032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "ac67e9740bbb4f4f9b1fa4beead80ff5"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "199900be3858292e32416d66654ccc3a"
server_key_iv: "947bb13131066c5589be66c4cc3fb9ab"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.290369 19032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:58.295058 19032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:58.295686 19032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:58.295869 19032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:58.296489 19032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:58.297433 19032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:58.297505 19032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:58.297573 19032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:58.297629 19032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:58.306919 19032 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:36231
I20260809 04:45:58.306948 19169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:36231 every 8 connection(s)
I20260809 04:45:58.307850 19032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:58.308902 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 19032
I20260809 04:45:58.308990 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:58.309314 7852 external_mini_cluster.cc:1428] Setting key 33b32a9412720304186b474c4f66e610
I20260809 04:45:58.311543 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43225
--builtin_ntp_servers=127.7.171.20:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 04:45:58.315639 19009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:58.310936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:38535 (local address 127.7.171.62:43225)
0809 04:45:58.311063 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:58.311067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:58.312489 (+ 1422us) server_negotiation.cc:407] Connection header received
0809 04:45:58.313829 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:58.313832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:58.313876 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:58.313979 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:58.315149 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:58.315153 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:58.315156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:58.315175 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:58.315196 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:58.315204 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:58.315301 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:58.315452 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 04:45:58.315499 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:45:58.316675 19172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:58.311444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43225 (local address 127.7.171.1:38535)
0809 04:45:58.312319 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:58.312373 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:58.313588 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:58.314158 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:58.314165 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:58.314254 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:58.314883 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:58.314884 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:58.314927 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:58.314996 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:58.315286 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:58.315295 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:58.315301 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:45:58.315489 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 04:45:58.315825 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":165,"threads_started":1}
I20260809 04:45:58.319420 19170 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43225
I20260809 04:45:58.319674 19170 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:58.320187 19170 heartbeater.cc:511] Master 127.7.171.62:43225 requested a full tablet report, sending...
I20260809 04:45:58.321759 18967 ts_manager.cc:194] Registered new tserver with Master: ac67e9740bbb4f4f9b1fa4beead80ff5 (127.7.171.1:36231)
W20260809 04:45:58.429661 19173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:58.430001 19173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:58.430079 19173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:58.434475 19173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:58.434590 19173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:58.434692 19173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:45:58.439801 19173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:58.441069 19173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:58.442659 19173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:58.453004 19182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:58.453078 19179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:58.453065 19173 server_base.cc:1034] running on GCE node
W20260809 04:45:58.453001 19178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:58.453804 19173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:58.454478 19173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:58.455674 19173 hybrid_clock.cc:648] HybridClock initialized: now 1786250758455664 us; error 36 us; skew 500 ppm
I20260809 04:45:58.457617 19173 webserver.cc:460] Webserver started at http://127.7.171.2:36709/ using document root <none> and password file <none>
I20260809 04:45:58.458246 19173 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:58.458345 19173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:58.458598 19173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:58.460922 19173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "85d9b13b1b4b4d438471a925370a854f"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "c0d4bcd2b1b7ea81d8ae21b84fd84cfd"
server_key_iv: "aaa7aa8a65c2bcb4e064ecb5e90f97c2"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.461508 19173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "85d9b13b1b4b4d438471a925370a854f"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "c0d4bcd2b1b7ea81d8ae21b84fd84cfd"
server_key_iv: "aaa7aa8a65c2bcb4e064ecb5e90f97c2"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.465378 19173 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:58.468488 19188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:58.470060 19173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:58.470211 19173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "85d9b13b1b4b4d438471a925370a854f"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "c0d4bcd2b1b7ea81d8ae21b84fd84cfd"
server_key_iv: "aaa7aa8a65c2bcb4e064ecb5e90f97c2"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.470340 19173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:58.475327 19173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:58.475966 19173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:58.476215 19173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:58.476810 19173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:58.477766 19173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:58.477841 19173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:58.477911 19173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:58.477963 19173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:58.487310 19173 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:44737
I20260809 04:45:58.487334 19314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:44737 every 8 connection(s)
I20260809 04:45:58.488545 19173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:58.489426 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 19173
I20260809 04:45:58.489531 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:58.489786 7852 external_mini_cluster.cc:1428] Setting key eafe96f89b9dc0abf2840b9265f266d7
I20260809 04:45:58.492118 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43225
--builtin_ntp_servers=127.7.171.20:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 04:45:58.496992 19009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:58.491537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:32881 (local address 127.7.171.62:43225)
0809 04:45:58.491735 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:58.491740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:58.493217 (+ 1477us) server_negotiation.cc:407] Connection header received
0809 04:45:58.495052 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:58.495058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:58.495107 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:58.495213 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:58.496560 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:58.496563 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:58.496565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:58.496577 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:58.496598 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:58.496606 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:58.496679 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:58.496848 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 04:45:58.496884 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:45:58.498301 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:58.492124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43225 (local address 127.7.171.2:32881)
0809 04:45:58.493012 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:58.493073 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:58.494817 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:58.495369 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:58.495376 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:58.495495 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:58.496229 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:58.496234 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:58.496293 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:58.496386 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:58.496693 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:58.496706 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:58.496715 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:45:58.496894 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 04:45:58.497173 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":142,"threads_started":1}
I20260809 04:45:58.499590 19317 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43225
I20260809 04:45:58.499848 19317 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:58.500387 19317 heartbeater.cc:511] Master 127.7.171.62:43225 requested a full tablet report, sending...
I20260809 04:45:58.501606 18967 ts_manager.cc:194] Registered new tserver with Master: 85d9b13b1b4b4d438471a925370a854f (127.7.171.2:44737)
W20260809 04:45:58.610987 19321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:58.611269 19321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:58.611311 19321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:58.615403 19321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:58.615477 19321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:58.615535 19321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:45:58.620417 19321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:45:58.621585 19321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:58.623013 19321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:58.634315 19327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:58.634315 19329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:58.634564 19321 server_base.cc:1034] running on GCE node
W20260809 04:45:58.634836 19326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:58.635251 19321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:58.635926 19321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:58.637130 19321 hybrid_clock.cc:648] HybridClock initialized: now 1786250758637113 us; error 29 us; skew 500 ppm
I20260809 04:45:58.638973 19321 webserver.cc:460] Webserver started at http://127.7.171.3:41559/ using document root <none> and password file <none>
I20260809 04:45:58.639575 19321 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:58.639669 19321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:58.639894 19321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:58.642227 19321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "a86c357289fb4004876b0f17ff6b3c1e"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "5674de9821538b9427484e1a291ac929"
server_key_iv: "8de4759cd99ccd9ddb0ae643ca8a1400"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.642819 19321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "a86c357289fb4004876b0f17ff6b3c1e"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "5674de9821538b9427484e1a291ac929"
server_key_iv: "8de4759cd99ccd9ddb0ae643ca8a1400"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.646643 19321 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:45:58.649387 19336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:58.650677 19321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:45:58.650821 19321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "a86c357289fb4004876b0f17ff6b3c1e"
format_stamp: "Formatted at 2026-08-09 04:45:58 on dist-test-slave-1zqn"
server_key: "5674de9821538b9427484e1a291ac929"
server_key_iv: "8de4759cd99ccd9ddb0ae643ca8a1400"
server_key_version: "encryptionkey@0"
I20260809 04:45:58.650945 19321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:58.655637 19321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:58.656347 19321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:58.656540 19321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:58.657125 19321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:58.658161 19321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:58.658248 19321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:58.658367 19321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:58.658419 19321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:58.667831 19321 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:39631
I20260809 04:45:58.667897 19463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:39631 every 8 connection(s)
I20260809 04:45:58.668910 19321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:58.670518 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 19321
I20260809 04:45:58.670616 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:58.670979 7852 external_mini_cluster.cc:1428] Setting key 7c5ef4b20b79a1be0d6264300330e303
I20260809 04:45:58.675870 19009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:58.670953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:32981 (local address 127.7.171.62:43225)
0809 04:45:58.671118 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:58.671124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:58.672618 (+ 1494us) server_negotiation.cc:407] Connection header received
0809 04:45:58.674262 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:58.674264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:58.674302 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:58.674371 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:58.675503 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:58.675505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:58.675507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:58.675517 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:58.675535 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:58.675542 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:58.675619 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:58.675728 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 04:45:58.675762 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:45:58.676856 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:58.671522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43225 (local address 127.7.171.3:32981)
0809 04:45:58.672474 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:58.672516 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:58.674064 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:58.674538 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:58.674545 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:58.674636 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:58.675262 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:58.675263 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:58.675305 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:58.675377 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:58.675626 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:58.675633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:58.675637 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:45:58.675771 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 04:45:58.676025 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":157,"threads_started":1}
I20260809 04:45:58.678023 19464 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43225
I20260809 04:45:58.678279 19464 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:58.678740 19464 heartbeater.cc:511] Master 127.7.171.62:43225 requested a full tablet report, sending...
I20260809 04:45:58.679915 18967 ts_manager.cc:194] Registered new tserver with Master: a86c357289fb4004876b0f17ff6b3c1e (127.7.171.3:39631)
I20260809 04:45:58.685163 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:45:58.688280 19009 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:58.687166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55588 (local address 127.7.171.62:43225)
0809 04:45:58.687305 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:58.687311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:58.687543 (+ 232us) server_negotiation.cc:407] Connection header received
0809 04:45:58.687730 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:58.687733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:58.687767 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:58.687849 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:58.688141 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55588: BlockingRecv error: recv got EOF from 127.0.0.1:55588 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 04:45:58.688619 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 19032
I20260809 04:45:58.696514 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 19173
I20260809 04:45:58.703831 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 19321
I20260809 04:45:58.711311 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 18930
2026-08-09T04:45:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (865 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:58 dist-test-slave-1zqn krb5kdc[19476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:58 dist-test-slave-1zqn krb5kdc[19476](info): set up 2 sockets
Aug 09 04:45:58 dist-test-slave-1zqn krb5kdc[19476](info): commencing operation
krb5kdc: starting...
W20260809 04:45:59.826104 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:45:59 dist-test-slave-1zqn krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:45:59.959537 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45493
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:45493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:00.074385 19490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:00.074717 19490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:00.074801 19490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:00.078938 19490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:46:00.079035 19490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:00.079072 19490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:00.079115 19490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:46:00.079147 19490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:46:00.084614 19490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:45493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:46:00.085757 19490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:00.086836 19490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:00.095587 19500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:00.095685 19490 server_base.cc:1034] running on GCE node
W20260809 04:46:00.095633 19498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:00.095599 19496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:00.096484 19490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:00.097561 19490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:00.098737 19490 hybrid_clock.cc:648] HybridClock initialized: now 1786250760098719 us; error 31 us; skew 500 ppm
Aug 09 04:46:00 dist-test-slave-1zqn krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250760, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:00.106349 19490 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:46:00.107261 19490 webserver.cc:460] Webserver started at http://127.7.171.62:39429/ using document root <none> and password file <none>
I20260809 04:46:00.107784 19490 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:00.107843 19490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:00.108014 19490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:00.110123 19490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "9fca2e4c6a964175a28f2fc19eb15b58"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "0c145ff1b757864f8e243d43abe9d35d"
server_key_iv: "2ceea06e86c57ec73fe8f734d41a0e86"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.110644 19490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "9fca2e4c6a964175a28f2fc19eb15b58"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "0c145ff1b757864f8e243d43abe9d35d"
server_key_iv: "2ceea06e86c57ec73fe8f734d41a0e86"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.114341 19490 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 04:46:00.116731 19507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:00.117962 19490 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:46:00.118094 19490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "9fca2e4c6a964175a28f2fc19eb15b58"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "0c145ff1b757864f8e243d43abe9d35d"
server_key_iv: "2ceea06e86c57ec73fe8f734d41a0e86"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.118183 19490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:00.146392 19490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:00.149394 19490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:00.149554 19490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:00.156857 19490 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:45493
I20260809 04:46:00.157200 19562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:45493 every 8 connection(s)
I20260809 04:46:00.157902 19490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:46:00.160791 19563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:00.166508 19563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58: Bootstrap starting.
I20260809 04:46:00.167411 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 19490
I20260809 04:46:00.167577 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:46:00.167851 7852 external_mini_cluster.cc:1428] Setting key 263e75db9d7dac65a40e176981c3f977
I20260809 04:46:00.169559 19563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:00.170470 19563 log.cc:824] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:00.172756 19563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58: No bootstrap required, opened a new log
I20260809 04:46:00.176923 19563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fca2e4c6a964175a28f2fc19eb15b58" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45493 } }
I20260809 04:46:00.177131 19563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:00.177191 19563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fca2e4c6a964175a28f2fc19eb15b58, State: Initialized, Role: FOLLOWER
Aug 09 04:46:00 dist-test-slave-1zqn krb5kdc[19476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:46:00.177714 19563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fca2e4c6a964175a28f2fc19eb15b58" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45493 } }
I20260809 04:46:00.177842 19563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:46:00.177925 19563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:46:00.178021 19563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:00.179322 19563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fca2e4c6a964175a28f2fc19eb15b58" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45493 } }
I20260809 04:46:00.179702 19563 leader_election.cc:302] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fca2e4c6a964175a28f2fc19eb15b58; no voters:
I20260809 04:46:00.180084 19563 leader_election.cc:288] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:46:00.180248 19568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:46:00.180477 19568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [term 1 LEADER]: Becoming Leader. State: Replica: 9fca2e4c6a964175a28f2fc19eb15b58, State: Running, Role: LEADER
I20260809 04:46:00.180882 19568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fca2e4c6a964175a28f2fc19eb15b58" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45493 } }
I20260809 04:46:00.181433 19563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:46:00.182955 19569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fca2e4c6a964175a28f2fc19eb15b58. Latest consensus state: current_term: 1 leader_uuid: "9fca2e4c6a964175a28f2fc19eb15b58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fca2e4c6a964175a28f2fc19eb15b58" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45493 } } }
I20260809 04:46:00.183390 19569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:00.183265 19570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fca2e4c6a964175a28f2fc19eb15b58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fca2e4c6a964175a28f2fc19eb15b58" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45493 } } }
I20260809 04:46:00.183857 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.169151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50972 (local address 127.7.171.62:45493)
0809 04:46:00.169664 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:00.169684 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:00.169728 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:46:00.170624 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:00.170647 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:00.171051 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:00.171404 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:00.172094 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.173554 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.174402 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.175120 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.175304 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.178095 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:00.178104 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:00.178121 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:00.178154 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:00.180175 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:00.180692 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:00.180698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:00.180705 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:00.180848 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:00.181199 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:00.181203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:00.181204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:00.181716 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:00.181942 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:00.182567 (+ 625us) server_negotiation.cc:299] Negotiation successful
0809 04:46:00.182875 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":147,"threads_started":1}
I20260809 04:46:00.184044 19570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58 [sys.catalog]: This master's current role is: LEADER
W20260809 04:46:00.187067 19584 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:46:00.187160 19584 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:46:00.187252 19583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:46:00.187975 19583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:46:00.193796 19583 catalog_manager.cc:1347] Generated new cluster ID: 25e93ad539ad439ab833f7877990f188
I20260809 04:46:00.193885 19583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:46:00.204389 19583 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:46:00.205487 19583 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:46:00.213536 19583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58: Generated new TSK 0
I20260809 04:46:00.214304 19583 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:46:00.252159 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45493
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:00.367930 19591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:00.368242 19591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:00.368288 19591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:00.372345 19591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:00.372414 19591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:00.372474 19591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:46:00.377745 19591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:46:00.378881 19591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:00.379894 19591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:00.390058 19599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:00.390035 19596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:00.390165 19591 server_base.cc:1034] running on GCE node
W20260809 04:46:00.390057 19597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:00.390790 19591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:00.391446 19591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:00.392614 19591 hybrid_clock.cc:648] HybridClock initialized: now 1786250760392599 us; error 27 us; skew 500 ppm
Aug 09 04:46:00 dist-test-slave-1zqn krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250760, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:00.395426 19591 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:46:00.396363 19591 webserver.cc:460] Webserver started at http://127.7.171.1:36993/ using document root <none> and password file <none>
I20260809 04:46:00.396924 19591 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:00.396987 19591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:00.397236 19591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:00.399289 19591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "111fc63b24704ae08203b4cb5dae7b61"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "2ad476f7d2ebc23fcf739d7b8ae90c2d"
server_key_iv: "b10f1dd00ad6dcd816b0619cb3fc5778"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.399858 19591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "111fc63b24704ae08203b4cb5dae7b61"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "2ad476f7d2ebc23fcf739d7b8ae90c2d"
server_key_iv: "b10f1dd00ad6dcd816b0619cb3fc5778"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.403669 19591 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:46:00.406261 19606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:00.407519 19591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:46:00.407648 19591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "111fc63b24704ae08203b4cb5dae7b61"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "2ad476f7d2ebc23fcf739d7b8ae90c2d"
server_key_iv: "b10f1dd00ad6dcd816b0619cb3fc5778"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.407763 19591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:00.430567 19591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:00.434027 19591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:00.434275 19591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:00.435016 19591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:00.436028 19591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:00.436185 19591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:00.436237 19591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:00.436254 19591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:00.445516 19591 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:46409
I20260809 04:46:00.445534 19734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:46409 every 8 connection(s)
I20260809 04:46:00.446468 19591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:46:00.450064 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 19591
I20260809 04:46:00.450202 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:46:00.450563 7852 external_mini_cluster.cc:1428] Setting key 00fe5cddf8c1e815e559b751a0c32607
Aug 09 04:46:00 dist-test-slave-1zqn krb5kdc[19476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250760, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:46:00.462703 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.448510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:48805 (local address 127.7.171.62:45493)
0809 04:46:00.448662 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:00.448666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:00.449579 (+ 913us) server_negotiation.cc:407] Connection header received
0809 04:46:00.451373 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:00.451376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:00.451432 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:00.451524 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:00.453170 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.453691 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.455145 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.455548 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.455697 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.458585 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:00.458592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:00.458598 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:00.458647 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:00.460395 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:00.460929 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:00.460933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:00.460938 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:00.461034 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:00.461384 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:00.461389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:00.461391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:00.461680 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:00.461798 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:00.462312 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 04:46:00.462498 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:46:00.463667 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.448911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45493 (local address 127.7.171.1:48805)
0809 04:46:00.449411 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:00.449456 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:00.451096 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:00.451688 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:00.451695 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:00.452276 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:00.452974 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.452997 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.453830 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.453836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.454997 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.455007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.455726 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.455729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.455871 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.456591 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:00.456613 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:00.458388 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:00.460528 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:00.460533 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:00.460547 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:00.460821 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:00.461135 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:00.461138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:00.461139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:00.461282 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:00.461801 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:00.461807 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:00.462051 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 04:46:00.462215 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 04:46:00.462568 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":125,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
I20260809 04:46:00.465247 19735 heartbeater.cc:347] Connected to a master server at 127.7.171.62:45493
I20260809 04:46:00.465533 19735 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:00.466173 19735 heartbeater.cc:511] Master 127.7.171.62:45493 requested a full tablet report, sending...
I20260809 04:46:00.467855 19525 ts_manager.cc:194] Registered new tserver with Master: 111fc63b24704ae08203b4cb5dae7b61 (127.7.171.1:46409)
I20260809 04:46:00.469456 19525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:48805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:46:00.483911 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45493
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:00.599654 19741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:00.599983 19741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:00.600035 19741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:00.604552 19741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:00.604627 19741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:00.604735 19741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:46:00.610232 19741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:46:00.611457 19741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:00.612646 19741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:00.622941 19746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:00.622941 19747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:00.622946 19749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:00.623483 19741 server_base.cc:1034] running on GCE node
I20260809 04:46:00.623946 19741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:00.624676 19741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:00.625841 19741 hybrid_clock.cc:648] HybridClock initialized: now 1786250760625803 us; error 52 us; skew 500 ppm
Aug 09 04:46:00 dist-test-slave-1zqn krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250760, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:00.628783 19741 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:46:00.629768 19741 webserver.cc:460] Webserver started at http://127.7.171.2:40603/ using document root <none> and password file <none>
I20260809 04:46:00.630350 19741 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:00.630452 19741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:00.630717 19741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:00.633569 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "912bfb825c0547c3af4fbfd8db03e928"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "734b8604b2ba2fa529b77a3b1fbc3e31"
server_key_iv: "2f3906e3012c4f0dcf9665b6f2464f7e"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.634383 19741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "912bfb825c0547c3af4fbfd8db03e928"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "734b8604b2ba2fa529b77a3b1fbc3e31"
server_key_iv: "2f3906e3012c4f0dcf9665b6f2464f7e"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.638293 19741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:46:00.640689 19758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:00.641947 19741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:46:00.642076 19741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "912bfb825c0547c3af4fbfd8db03e928"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "734b8604b2ba2fa529b77a3b1fbc3e31"
server_key_iv: "2f3906e3012c4f0dcf9665b6f2464f7e"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.642180 19741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:00.658402 19741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:00.661305 19741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:00.661532 19741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:00.662137 19741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:00.662976 19741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:00.663021 19741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:00.663086 19741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:00.663121 19741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:00.672118 19741 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:42575
I20260809 04:46:00.672145 19883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:42575 every 8 connection(s)
I20260809 04:46:00.673059 19741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:46:00.681929 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 19741
I20260809 04:46:00.682075 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:46:00.682533 7852 external_mini_cluster.cc:1428] Setting key 5961ac2e9890058f039d50113596141b
Aug 09 04:46:00 dist-test-slave-1zqn krb5kdc[19476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250760, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:46:00.692255 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.675108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:58201 (local address 127.7.171.62:45493)
0809 04:46:00.675291 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:00.675296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:00.676224 (+ 928us) server_negotiation.cc:407] Connection header received
0809 04:46:00.677663 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:00.677666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:00.677718 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:00.677804 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:00.679163 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.679713 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.681083 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.681327 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.681487 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.685261 (+ 3774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:00.685272 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:00.685277 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:00.685304 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:00.689370 (+ 4066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:00.690159 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:00.690162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:00.690164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:00.690277 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:00.690719 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:00.690724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:00.690729 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:00.690973 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:00.691049 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:00.691844 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 04:46:00.692006 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:46:00.693429 19886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.675433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45493 (local address 127.7.171.2:58201)
0809 04:46:00.675999 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:00.676048 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:00.677429 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:00.678057 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:00.678064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:00.678467 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:00.678995 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.679011 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.679945 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.679949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.680945 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.680955 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.681510 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.681514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.681727 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.682665 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:00.682688 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:00.684802 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:00.689529 (+ 4727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:00.689536 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:00.689554 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:00.690022 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:00.690395 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:00.690400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:00.690405 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:00.690606 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:00.691126 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:00.691134 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:00.691488 (+ 354us) client_negotiation.cc:769] Sending connection context
0809 04:46:00.691746 (+ 258us) client_negotiation.cc:240] Negotiation successful
0809 04:46:00.692202 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":109,"threads_started":1}
I20260809 04:46:00.695068 19884 heartbeater.cc:347] Connected to a master server at 127.7.171.62:45493
I20260809 04:46:00.695375 19884 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:00.696099 19884 heartbeater.cc:511] Master 127.7.171.62:45493 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:46:00.697552 19525 ts_manager.cc:194] Registered new tserver with Master: 912bfb825c0547c3af4fbfd8db03e928 (127.7.171.2:42575)
I20260809 04:46:00.698559 19525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:58201
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:46:00.719990 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45493
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:00.839931 19889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:00.840274 19889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:00.840320 19889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:00.844476 19889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:00.844550 19889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:00.844660 19889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:46:00.850395 19889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:45493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:46:00.851531 19889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:00.852610 19889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:00.862696 19897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:00.862718 19895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:00.862761 19889 server_base.cc:1034] running on GCE node
W20260809 04:46:00.862793 19894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:00.863344 19889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:00.864001 19889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:00.865217 19889 hybrid_clock.cc:648] HybridClock initialized: now 1786250760865205 us; error 45 us; skew 500 ppm
Aug 09 04:46:00 dist-test-slave-1zqn krb5kdc[19476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250760, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:00.867965 19889 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:46:00.868865 19889 webserver.cc:460] Webserver started at http://127.7.171.3:33745/ using document root <none> and password file <none>
I20260809 04:46:00.869441 19889 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:00.869503 19889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:00.869763 19889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:00.871801 19889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "d56a5fba48644ea2b11e6ce18d15406a"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "7f805f4897839167adac928ef9be161d"
server_key_iv: "03aae4ec7a4439fe6395edc17094c54b"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.872464 19889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "d56a5fba48644ea2b11e6ce18d15406a"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "7f805f4897839167adac928ef9be161d"
server_key_iv: "03aae4ec7a4439fe6395edc17094c54b"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.876302 19889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:46:00.878753 19905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:00.879940 19889 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:46:00.880132 19889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "d56a5fba48644ea2b11e6ce18d15406a"
format_stamp: "Formatted at 2026-08-09 04:46:00 on dist-test-slave-1zqn"
server_key: "7f805f4897839167adac928ef9be161d"
server_key_iv: "03aae4ec7a4439fe6395edc17094c54b"
server_key_version: "encryptionkey@0"
I20260809 04:46:00.880231 19889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:00.892278 19889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:00.895182 19889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:00.895390 19889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:00.896018 19889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:00.896906 19889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:00.896955 19889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:00.897025 19889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:00.897065 19889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:00.906273 19889 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:35995
I20260809 04:46:00.906296 20028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:35995 every 8 connection(s)
I20260809 04:46:00.907199 19889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:46:00.908946 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 19889
I20260809 04:46:00.909085 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:46:00.909430 7852 external_mini_cluster.cc:1428] Setting key 55aa7562bda9bb4d8786b8a4d3943c37
Aug 09 04:46:00 dist-test-slave-1zqn krb5kdc[19476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250760, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:46:00.922176 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.909506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:57883 (local address 127.7.171.62:45493)
0809 04:46:00.909660 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:00.909665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:00.910757 (+ 1092us) server_negotiation.cc:407] Connection header received
0809 04:46:00.912476 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:00.912479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:00.912532 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:00.912603 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:00.913779 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.914332 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.915453 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.915684 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.915765 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.918618 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:00.918621 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:00.918623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:00.918661 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:00.920203 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:00.920735 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:00.920738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:00.920739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:00.920812 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:00.921115 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:00.921118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:00.921119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:00.921294 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:00.921417 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:00.921930 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 04:46:00.922049 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:46:00.923416 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.909980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:45493 (local address 127.7.171.3:57883)
0809 04:46:00.910608 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:00.910652 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:00.912277 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:00.912768 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:00.912775 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:00.913143 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:00.913632 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.913645 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.914461 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.914464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.915342 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.915351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.916037 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.916041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.916239 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.916818 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:00.916835 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:00.918452 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:00.920309 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:00.920314 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:00.920326 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:00.920616 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:00.920892 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:00.920894 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:00.920896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:00.921035 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:00.921431 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:00.921436 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:00.921678 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 04:46:00.921890 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 04:46:00.922235 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":144,"threads_started":1}
I20260809 04:46:00.924654 20029 heartbeater.cc:347] Connected to a master server at 127.7.171.62:45493
I20260809 04:46:00.924875 20029 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:00.925329 20029 heartbeater.cc:511] Master 127.7.171.62:45493 requested a full tablet report, sending...
I20260809 04:46:00.926326 19525 ts_manager.cc:194] Registered new tserver with Master: d56a5fba48644ea2b11e6ce18d15406a (127.7.171.3:35995)
I20260809 04:46:00.926931 19525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:57883
I20260809 04:46:00.935720 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:46:00.944891 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.937698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45854 (local address 127.7.171.62:45493)
0809 04:46:00.937846 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:00.937851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:00.938045 (+ 194us) server_negotiation.cc:407] Connection header received
0809 04:46:00.938160 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:00.938162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:00.938203 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:00.938270 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:00.938958 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.939468 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.940328 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.940535 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.940609 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.941867 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:00.941870 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:00.941872 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:00.941894 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:00.943336 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:00.943688 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:00.943690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:00.943691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:00.943764 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:00.943992 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:00.943994 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:00.943995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:00.944271 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:00.944373 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:00.944571 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 04:46:00.944666 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:46:00.947739 19525 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:46:00.950766 19525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:46:00.965368 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.959147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:42575 (local address 127.0.0.1:43518)
0809 04:46:00.959898 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:00.959951 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:00.960140 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:00.960717 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:00.960724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:00.960754 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:00.960912 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.960922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.962367 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.962371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.963714 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.963727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.964807 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.964811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.964925 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.964970 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 04:46:00.965074 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 04:46:00.965198 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":83,"threads_started":1}
I20260809 04:46:00.965572 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.960182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35686 (local address 127.7.171.1:46409)
0809 04:46:00.960811 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:00.960826 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:00.960850 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:46:00.960923 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:00.960928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:00.961115 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:00.961271 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:00.961469 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.962510 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.964008 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.964955 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.965076 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.965222 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:00.965319 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 04:46:00.965420 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":96,"threads_started":1}
I20260809 04:46:00.965826 20043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.959205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:35995 (local address 127.0.0.1:33692)
0809 04:46:00.960344 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:00.960358 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:00.960446 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:00.961616 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:00.961619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:00.961637 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:00.961734 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.961741 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.962963 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.962967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.964128 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.964137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.965411 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.965415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.965507 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.965540 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 04:46:00.965610 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 04:46:00.965691 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"mutex_wait_us":26,"spinlock_wait_cycles":3584,"thread_start_us":109,"threads_started":1}
I20260809 04:46:00.965977 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.959268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43518 (local address 127.7.171.2:42575)
0809 04:46:00.959597 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:00.959609 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:00.960037 (+ 428us) server_negotiation.cc:407] Connection header received
0809 04:46:00.960323 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:00.960329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:00.960544 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:00.960712 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:00.961015 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.962224 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.963852 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.964679 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.964790 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.964944 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:00.965564 (+ 620us) server_negotiation.cc:299] Negotiation successful
0809 04:46:00.965744 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":127,"threads_started":1}
I20260809 04:46:00.966259 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.959886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:46409 (local address 127.0.0.1:35686)
0809 04:46:00.960344 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:00.960358 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:00.960446 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:00.961261 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:00.961264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:00.961283 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:00.961370 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.961375 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.962648 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.962653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.963872 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:00.963882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.965070 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.965074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:00.965182 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.965208 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:46:00.965292 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:46:00.965366 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":23,"spinlock_wait_cycles":6656,"thread_start_us":101,"threads_started":1}
I20260809 04:46:00.966292 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:00.959958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33692 (local address 127.7.171.3:35995)
0809 04:46:00.961143 (+ 1185us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:00.961152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:00.961170 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:46:00.961261 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:00.961266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:00.961454 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:00.961638 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:00.961867 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.962810 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.964261 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:00.965103 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:00.965203 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:00.965349 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:00.966025 (+ 676us) server_negotiation.cc:299] Negotiation successful
0809 04:46:00.966149 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1094,"thread_start_us":497,"threads_started":1}
I20260809 04:46:00.968912 19813 tablet_service.cc:1467] Processing CreateTablet for tablet 9f3058e900b34ec8a3a869ef76a041ba (DEFAULT_TABLE table=test-table [id=df4bc23135484e19923ac7c0f2a47567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:00.969080 19955 tablet_service.cc:1467] Processing CreateTablet for tablet 9f3058e900b34ec8a3a869ef76a041ba (DEFAULT_TABLE table=test-table [id=df4bc23135484e19923ac7c0f2a47567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:00.969221 19663 tablet_service.cc:1467] Processing CreateTablet for tablet 9f3058e900b34ec8a3a869ef76a041ba (DEFAULT_TABLE table=test-table [id=df4bc23135484e19923ac7c0f2a47567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:00.969995 19813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f3058e900b34ec8a3a869ef76a041ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:00.970137 19955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f3058e900b34ec8a3a869ef76a041ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:00.970131 19663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f3058e900b34ec8a3a869ef76a041ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:00.976089 20048 tablet_bootstrap.cc:492] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928: Bootstrap starting.
I20260809 04:46:00.978309 20049 tablet_bootstrap.cc:492] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61: Bootstrap starting.
I20260809 04:46:00.978888 20048 tablet_bootstrap.cc:654] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:00.979647 20048 log.cc:824] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:00.980338 20050 tablet_bootstrap.cc:492] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: Bootstrap starting.
I20260809 04:46:00.980427 20049 tablet_bootstrap.cc:654] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:00.981108 20049 log.cc:824] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:00.982069 20050 tablet_bootstrap.cc:654] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:00.983455 20049 tablet_bootstrap.cc:492] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61: No bootstrap required, opened a new log
I20260809 04:46:00.983659 20049 ts_tablet_manager.cc:1397] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:46:00.983867 20048 tablet_bootstrap.cc:492] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928: No bootstrap required, opened a new log
I20260809 04:46:00.984122 20048 ts_tablet_manager.cc:1397] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 04:46:00.987057 20048 raft_consensus.cc:348] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "912bfb825c0547c3af4fbfd8db03e928" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42575 } } peers { permanent_uuid: "d56a5fba48644ea2b11e6ce18d15406a" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 35995 } } peers { permanent_uuid: "111fc63b24704ae08203b4cb5dae7b61" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46409 } }
I20260809 04:46:00.987097 20049 raft_consensus.cc:348] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "912bfb825c0547c3af4fbfd8db03e928" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42575 } } peers { permanent_uuid: "d56a5fba48644ea2b11e6ce18d15406a" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 35995 } } peers { permanent_uuid: "111fc63b24704ae08203b4cb5dae7b61" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46409 } }
I20260809 04:46:00.987277 20048 raft_consensus.cc:374] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:00.987313 20049 raft_consensus.cc:374] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:00.987342 20048 raft_consensus.cc:729] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 912bfb825c0547c3af4fbfd8db03e928, State: Initialized, Role: FOLLOWER
I20260809 04:46:00.987381 20049 raft_consensus.cc:729] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 111fc63b24704ae08203b4cb5dae7b61, State: Initialized, Role: FOLLOWER
I20260809 04:46:00.987823 20048 consensus_queue.cc:260] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "912bfb825c0547c3af4fbfd8db03e928" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42575 } } peers { permanent_uuid: "d56a5fba48644ea2b11e6ce18d15406a" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 35995 } } peers { permanent_uuid: "111fc63b24704ae08203b4cb5dae7b61" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46409 } }
I20260809 04:46:00.987864 20049 consensus_queue.cc:260] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "912bfb825c0547c3af4fbfd8db03e928" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42575 } } peers { permanent_uuid: "d56a5fba48644ea2b11e6ce18d15406a" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 35995 } } peers { permanent_uuid: "111fc63b24704ae08203b4cb5dae7b61" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46409 } }
I20260809 04:46:00.988480 20050 log.cc:824] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:00.988858 20048 ts_tablet_manager.cc:1428] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260809 04:46:00.988914 20049 ts_tablet_manager.cc:1428] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 04:46:00.989265 19735 heartbeater.cc:503] Master 127.7.171.62:45493 was elected leader, sending a full tablet report...
I20260809 04:46:00.989253 19884 heartbeater.cc:503] Master 127.7.171.62:45493 was elected leader, sending a full tablet report...
I20260809 04:46:00.990504 20050 tablet_bootstrap.cc:492] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: No bootstrap required, opened a new log
I20260809 04:46:00.990731 20050 ts_tablet_manager.cc:1397] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.005s
I20260809 04:46:00.994652 20050 raft_consensus.cc:348] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "912bfb825c0547c3af4fbfd8db03e928" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42575 } } peers { permanent_uuid: "d56a5fba48644ea2b11e6ce18d15406a" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 35995 } } peers { permanent_uuid: "111fc63b24704ae08203b4cb5dae7b61" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46409 } }
I20260809 04:46:00.994903 20050 raft_consensus.cc:374] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:00.994967 20050 raft_consensus.cc:729] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d56a5fba48644ea2b11e6ce18d15406a, State: Initialized, Role: FOLLOWER
I20260809 04:46:00.995397 20050 consensus_queue.cc:260] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "912bfb825c0547c3af4fbfd8db03e928" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42575 } } peers { permanent_uuid: "d56a5fba48644ea2b11e6ce18d15406a" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 35995 } } peers { permanent_uuid: "111fc63b24704ae08203b4cb5dae7b61" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46409 } }
I20260809 04:46:00.996162 20029 heartbeater.cc:503] Master 127.7.171.62:45493 was elected leader, sending a full tablet report...
I20260809 04:46:00.996488 20050 ts_tablet_manager.cc:1428] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260809 04:46:01.158316 20031 tablet.cc:2369] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:46:01.172282 20054 raft_consensus.cc:482] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:46:01.172513 20054 raft_consensus.cc:504] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "912bfb825c0547c3af4fbfd8db03e928" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42575 } } peers { permanent_uuid: "d56a5fba48644ea2b11e6ce18d15406a" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 35995 } } peers { permanent_uuid: "111fc63b24704ae08203b4cb5dae7b61" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46409 } }
I20260809 04:46:01.173740 20054 leader_election.cc:288] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d56a5fba48644ea2b11e6ce18d15406a (127.7.171.3:35995), 111fc63b24704ae08203b4cb5dae7b61 (127.7.171.1:46409)
W20260809 04:46:01.174245 19885 tablet.cc:2369] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:46:01.177953 19886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:01.174032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:35995 (local address 127.7.171.2:59049)
0809 04:46:01.174291 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:01.174307 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:01.174412 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:01.174757 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:01.174760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:01.174776 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:01.174904 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:01.174911 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:01.175855 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:01.175858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:01.176754 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:01.176765 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:01.177516 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:01.177519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:01.177644 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:01.177678 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 04:46:01.177754 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 04:46:01.177831 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260809 04:46:01.177990 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:01.174246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:59049 (local address 127.7.171.3:35995)
0809 04:46:01.174394 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:01.174399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:01.174419 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:46:01.174567 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:01.174571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:01.174633 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:01.174744 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:01.175065 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:01.175736 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:01.176888 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:01.177392 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:01.177474 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:01.177536 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:01.177785 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 04:46:01.177857 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:46:01.178269 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:01.174191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:51397 (local address 127.7.171.1:46409)
0809 04:46:01.174343 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:01.174349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:01.174773 (+ 424us) server_negotiation.cc:407] Connection header received
0809 04:46:01.175012 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:01.175015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:01.175084 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:01.175142 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:01.175442 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:01.175997 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:01.177160 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:01.177642 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:01.177756 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:01.177814 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:01.178029 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 04:46:01.178125 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:46:01.178551 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:01.174032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:46409 (local address 127.7.171.2:51397)
0809 04:46:01.174661 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:01.174675 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:01.174870 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:01.175188 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:01.175191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:01.175216 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:01.175331 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:01.175338 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:01.176161 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:01.176164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:01.177035 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:01.177046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:01.177787 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:01.177792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:01.177888 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:01.177911 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:46:01.177988 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 04:46:01.178412 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":327,"thread_start_us":127,"threads_started":1}
I20260809 04:46:01.178718 19977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3058e900b34ec8a3a869ef76a041ba" candidate_uuid: "912bfb825c0547c3af4fbfd8db03e928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d56a5fba48644ea2b11e6ce18d15406a" is_pre_election: true
I20260809 04:46:01.179064 19977 raft_consensus.cc:2454] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 912bfb825c0547c3af4fbfd8db03e928 in term 0.
I20260809 04:46:01.179201 19684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3058e900b34ec8a3a869ef76a041ba" candidate_uuid: "912bfb825c0547c3af4fbfd8db03e928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "111fc63b24704ae08203b4cb5dae7b61" is_pre_election: true
I20260809 04:46:01.179492 19684 raft_consensus.cc:2454] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 912bfb825c0547c3af4fbfd8db03e928 in term 0.
I20260809 04:46:01.179561 19762 leader_election.cc:302] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 912bfb825c0547c3af4fbfd8db03e928, d56a5fba48644ea2b11e6ce18d15406a; no voters:
I20260809 04:46:01.179832 20054 raft_consensus.cc:2781] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:46:01.179922 20054 raft_consensus.cc:482] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:46:01.179975 20054 raft_consensus.cc:3037] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:01.181057 20054 raft_consensus.cc:504] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "912bfb825c0547c3af4fbfd8db03e928" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42575 } } peers { permanent_uuid: "d56a5fba48644ea2b11e6ce18d15406a" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 35995 } } peers { permanent_uuid: "111fc63b24704ae08203b4cb5dae7b61" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46409 } }
I20260809 04:46:01.181448 20054 leader_election.cc:288] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [CANDIDATE]: Term 1 election: Requested vote from peers d56a5fba48644ea2b11e6ce18d15406a (127.7.171.3:35995), 111fc63b24704ae08203b4cb5dae7b61 (127.7.171.1:46409)
I20260809 04:46:01.181756 19977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3058e900b34ec8a3a869ef76a041ba" candidate_uuid: "912bfb825c0547c3af4fbfd8db03e928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d56a5fba48644ea2b11e6ce18d15406a"
I20260809 04:46:01.181871 19684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f3058e900b34ec8a3a869ef76a041ba" candidate_uuid: "912bfb825c0547c3af4fbfd8db03e928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "111fc63b24704ae08203b4cb5dae7b61"
I20260809 04:46:01.181965 19977 raft_consensus.cc:3037] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:01.182013 19684 raft_consensus.cc:3037] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:01.183055 19977 raft_consensus.cc:2454] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 912bfb825c0547c3af4fbfd8db03e928 in term 1.
I20260809 04:46:01.183058 19684 raft_consensus.cc:2454] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 912bfb825c0547c3af4fbfd8db03e928 in term 1.
I20260809 04:46:01.183387 19761 leader_election.cc:302] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 111fc63b24704ae08203b4cb5dae7b61, 912bfb825c0547c3af4fbfd8db03e928; no voters:
I20260809 04:46:01.183569 20054 raft_consensus.cc:2781] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:46:01.183833 20054 raft_consensus.cc:686] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 1 LEADER]: Becoming Leader. State: Replica: 912bfb825c0547c3af4fbfd8db03e928, State: Running, Role: LEADER
I20260809 04:46:01.184216 20054 consensus_queue.cc:237] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "912bfb825c0547c3af4fbfd8db03e928" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42575 } } peers { permanent_uuid: "d56a5fba48644ea2b11e6ce18d15406a" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 35995 } } peers { permanent_uuid: "111fc63b24704ae08203b4cb5dae7b61" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46409 } }
I20260809 04:46:01.187469 19525 catalog_manager.cc:5495] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 reported cstate change: term changed from 0 to 1, leader changed from <none> to 912bfb825c0547c3af4fbfd8db03e928 (127.7.171.2). New cstate: current_term: 1 leader_uuid: "912bfb825c0547c3af4fbfd8db03e928" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "912bfb825c0547c3af4fbfd8db03e928" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d56a5fba48644ea2b11e6ce18d15406a" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 35995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "111fc63b24704ae08203b4cb5dae7b61" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46409 } health_report { overall_health: UNKNOWN } } }
W20260809 04:46:01.197775 19736 tablet.cc:2369] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:46:01.207839 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:01.204164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43534 (local address 127.7.171.2:42575)
0809 04:46:01.204300 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:01.204304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:01.204317 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:46:01.204395 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:01.204398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:01.204453 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:01.204553 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:46:01.204848 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:01.205472 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:01.206355 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:01.206581 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:01.206689 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:01.206981 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:46:01.207447 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:46:01.207550 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:01.207627 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 04:46:01.207685 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:46:01.214185 19684 raft_consensus.cc:1261] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61 [term 1 FOLLOWER]: Refusing update from remote peer 912bfb825c0547c3af4fbfd8db03e928: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:46:01.214185 19977 raft_consensus.cc:1261] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a [term 1 FOLLOWER]: Refusing update from remote peer 912bfb825c0547c3af4fbfd8db03e928: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:46:01.214988 20060 consensus_queue.cc:1035] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d56a5fba48644ea2b11e6ce18d15406a" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:46:01.215229 20054 consensus_queue.cc:1035] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [LEADER]: Connected to new peer: Peer: permanent_uuid: "111fc63b24704ae08203b4cb5dae7b61" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:46:01.224496 20064 mvcc.cc:205] Tried to move back new op lower bound from 7316483117925888000 to 7316483117813608448. Current Snapshot: MvccSnapshot[applied={T|T < 7316483117925888000}]
I20260809 04:46:01.227828 20063 mvcc.cc:205] Tried to move back new op lower bound from 7316483117925888000 to 7316483117813608448. Current Snapshot: MvccSnapshot[applied={T|T < 7316483117925888000 or (T in {7316483117925888000})}]
I20260809 04:46:01.233016 19525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:46:01.233197 19525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:46:01.235704 19525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fca2e4c6a964175a28f2fc19eb15b58: Sending DeleteTablet for 3 replicas of tablet 9f3058e900b34ec8a3a869ef76a041ba
I20260809 04:46:01.236549 19955 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f3058e900b34ec8a3a869ef76a041ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:01 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:33692
I20260809 04:46:01.236557 19813 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f3058e900b34ec8a3a869ef76a041ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:01 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:43518
I20260809 04:46:01.237110 20075 tablet_replica.cc:330] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: stopping tablet replica
I20260809 04:46:01.237496 20075 raft_consensus.cc:2229] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:46:01.237462 19663 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f3058e900b34ec8a3a869ef76a041ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:01 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:35686
I20260809 04:46:01.237871 20075 raft_consensus.cc:2258] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:46:01.237975 20077 tablet_replica.cc:330] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61: stopping tablet replica
I20260809 04:46:01.237978 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 19591
I20260809 04:46:01.238281 20077 raft_consensus.cc:2229] T 9f3058e900b34ec8a3a869ef76a041ba P 111fc63b24704ae08203b4cb5dae7b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:46:01.243108 20075 ts_tablet_manager.cc:1905] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:46:01.244333 20076 tablet_replica.cc:330] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928: stopping tablet replica
I20260809 04:46:01.244758 20076 raft_consensus.cc:2229] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:46:01.245322 20076 raft_consensus.cc:2258] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:46:01.246922 20075 ts_tablet_manager.cc:1918] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:46:01.247041 20075 log.cc:1196] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/9f3058e900b34ec8a3a869ef76a041ba
I20260809 04:46:01.247429 20075 ts_tablet_manager.cc:1939] T 9f3058e900b34ec8a3a869ef76a041ba P d56a5fba48644ea2b11e6ce18d15406a: Deleting consensus metadata
I20260809 04:46:01.247450 20076 ts_tablet_manager.cc:1905] T 9f3058e900b34ec8a3a869ef76a041ba P 912bfb825c0547c3af4fbfd8db03e928: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:46:01.248884 19511 catalog_manager.cc:4841] TS d56a5fba48644ea2b11e6ce18d15406a (127.7.171.3:35995): tablet 9f3058e900b34ec8a3a869ef76a041ba (table test-table [id=df4bc23135484e19923ac7c0f2a47567]) successfully deleted
W20260809 04:46:01.249156 19510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:46409 (error 108)
I20260809 04:46:01.250257 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:01.249924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:46409 (local address 127.0.0.1:35702)
0809 04:46:01.250061 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:01.250152 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:46409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 04:46:01.250418 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 19741
W20260809 04:46:01.250592 19510 catalog_manager.cc:4584] TS 111fc63b24704ae08203b4cb5dae7b61 (127.7.171.1:46409): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9f3058e900b34ec8a3a869ef76a041ba: Network error: Client connection negotiation failed: client connection to 127.7.171.1:46409: connect: Connection refused (error 111)
I20260809 04:46:01.258824 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 19889
I20260809 04:46:01.259262 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:01.258949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:42575 (local address 127.0.0.1:43536)
0809 04:46:01.259090 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:01.259157 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.2:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 04:46:01.267226 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 19490
2026-08-09T04:46:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2556 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:46:01 dist-test-slave-1zqn krb5kdc[20081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:46:01 dist-test-slave-1zqn krb5kdc[20081](info): set up 2 sockets
Aug 09 04:46:01 dist-test-slave-1zqn krb5kdc[20081](info): commencing operation
krb5kdc: starting...
W20260809 04:46:02.397135 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:46:02 dist-test-slave-1zqn krb5kdc[20081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:46:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:46:02Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.171.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.171.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/rangeradmin_127.7.171.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/rangeradmin_127.7.171.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.171.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.171.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/rangerlookup_127.7.171.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/rangerlookup_127.7.171.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.171.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.171.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/HTTP_127.7.171.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/HTTP_127.7.171.20@KRBTEST.COM.keytab.
I20260809 04:46:02.590997 7852 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfTSqH0/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/postgres -l logfile start
2026-08-09 04:46:05.260 UTC [20118] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 04:46:05.260 UTC [20118] LOG: listening on IPv4 address "127.7.171.20", port 58365
2026-08-09 04:46:05.264 UTC [20118] LOG: listening on Unix socket "/tmp/.s.PGSQL.58365"
2026-08-09 04:46:05.301 UTC [20121] LOG: database system was shut down at 2026-08-09 04:46:03 UTC
2026-08-09 04:46:05.306 UTC [20118] LOG: database system is ready to accept connections
I20260809 04:46:06.306725 7852 mini_postgres.cc:96] Postgres bound to 58365
2026-08-09 04:46:06.314 UTC [20130] FATAL: database "slave" does not exist
127.7.171.20:58365 - accepting connections
I20260809 04:46:06.315671 7852 mini_ranger.cc:161] Starting Ranger...
I20260809 04:46:06.332006 7852 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 04:46:06.864912 7852 mini_ranger.cc:87] Created ranger Postgres database
I20260809 04:46:06.865079 7852 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-admin
2026-08-09 04:46:07,211 [I] DB FLAVOR :POSTGRES
2026-08-09 04:46:07,212 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:46:07,212 [I] Checking connection..
2026-08-09 04:46:07,212 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 04:46:07,484 [I] Checking connection passed.
2026-08-09 04:46:07,485 [I] --------- Verifying version history table ---------
2026-08-09 04:46:07,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:46:07,746 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:46:07,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:46:07,996 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:46:07,997 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 04:46:07,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 04:46:08.265 UTC [20201] WARNING: there is no transaction in progress
2026-08-09 04:46:08,272 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 04:46:08,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:46:08,544 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 04:46:08,544 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:46:08,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 04:46:08,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 04:46:09,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1zqn.c.gcp-upstream.internal','N') ;"
2026-08-09 04:46:09,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:46:09,586 [I] Table x_portal_user does not exist in database ranger
2026-08-09 04:46:09,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:46:09,836 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 04:46:09,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:46:10,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:46:10,354 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 04:46:10,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 04:46:11.339 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.356 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.373 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.389 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.406 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.424 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.483 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.490 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.505 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.516 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.526 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.543 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.553 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.560 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.567 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:11.574 UTC [20325] WARNING: there is no transaction in progress
2026-08-09 04:46:12,262 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 04:46:12,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 04:46:12,527 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 04:46:12,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:46:12,783 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 04:46:12,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:46:13,038 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 04:46:13,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:46:13,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:46:13,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1zqn.c.gcp-upstream.internal';"
2026-08-09 04:46:13,804 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 04:46:13,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 04:46:14,055 [I] DB_PATCHES have already been applied
I20260809 04:46:14.061944 7852 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfTSqH0/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 04:46:14.062037 7852 mini_ranger.cc:193] Using host: 127.7.171.20
I20260809 04:46:14.066502 7852 mini_ranger.cc:227] Ranger admin URL: http://127.7.171.20:51317
Aug 09, 2026 4:46:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:46:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:46:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:46:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:46:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:46:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:46:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 04:46:15.223774 7852 mini_ranger.cc:160] Time spent starting Ranger: real 8.908s user 0.000s sys 0.013s
Aug 09, 2026 4:46:36 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:46:36 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 4:46:36 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:46:36 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:46:37 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 04:46:37.619730 7852 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.171.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.171.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/rangerkms_127.7.171.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/rangerkms_127.7.171.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.171.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.171.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.171.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/HTTP_127.7.171.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.171.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/HTTP_127.7.171.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 04:46:37 dist-test-slave-1zqn krb5kdc[20081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250797, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 04:46:37.729353 7852 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 04:46:37.740145 7852 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 04:46:38.682668 7852 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 04:46:38.682781 7852 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-kms
I20260809 04:46:38.682901 7852 mini_ranger_kms.cc:227] Using postgres at 127.7.171.20:58365
2026-08-09 04:46:38,919 [I] DB FLAVOR :POSTGRES
2026-08-09 04:46:38,919 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:46:38,919 [I] Checking connection
2026-08-09 04:46:38,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 04:46:39,195 [I] connection success
2026-08-09 04:46:39,195 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 04:46:39,196 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 04:46:39,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 04:46:39,446 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 04:46:39,446 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:46:39,446 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 04:46:39,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:58365/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 04:46:39,712 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 04:46:40.212455 7852 mini_ranger_kms.cc:310] Created kms service
I20260809 04:46:40.430917 7852 mini_ranger_kms.cc:326] Created kudu user
I20260809 04:46:40.476099 7852 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 04:46:40 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250797, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:46:40.866873 7852 mini_ranger_kms.cc:384] Added ranger policy
I20260809 04:46:40.867031 7852 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 04:46:40.867055 7852 mini_ranger_kms.cc:240] Using host: 127.7.171.20
I20260809 04:46:40.872311 7852 mini_ranger_kms.cc:276] Ranger KMS PID: 20531
I20260809 04:46:40.872485 7852 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.171.20:59315
Aug 09, 2026 4:46:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:46:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:46:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:46:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:46:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:46:42 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 04:46:43.315004 7852 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.586s user 0.020s sys 0.202s
I20260809 04:46:43.315141 7852 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 04:46:43.315258 7852 mini_ranger_kms.cc:401] 127.7.171.20:59315/kms/v1/keys
Aug 09 04:46:43 dist-test-slave-1zqn krb5kdc[20081](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250803, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.171.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 04:46:43 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250803, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.171.20@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
Aug 09 04:46:44 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250803, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.171.20@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
Aug 09 04:46:46 dist-test-slave-1zqn krb5kdc[20081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:46:46.679286 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35357
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:35357
--ranger_config_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:59315/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:46.796921 20594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:46.797187 20594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:46.797240 20594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:46.801173 20594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:46:46.801239 20594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:46.801257 20594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:46.801275 20594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:46:46.801290 20594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:46:46.808329 20594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.171.20:59315/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:35357
--ranger_config_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:46:46.809485 20594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:46.810498 20594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:46.820043 20603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:46.820158 20594 server_base.cc:1034] running on GCE node
W20260809 04:46:46.820043 20605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:46.820158 20602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:46.820991 20594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:46.822135 20594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:46.823323 20594 hybrid_clock.cc:648] HybridClock initialized: now 1786250806823308 us; error 29 us; skew 500 ppm
Aug 09 04:46:46 dist-test-slave-1zqn krb5kdc[20081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250806, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:46.830627 20594 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:46:46.831597 20594 webserver.cc:460] Webserver started at http://127.7.171.62:38581/ using document root <none> and password file <none>
I20260809 04:46:46.832231 20594 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:46.832315 20594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:46.832531 20594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:46:46 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250806, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:46:46.884483 20594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "0afbf7315d8c44d586ac52db06e8093f"
format_stamp: "Formatted at 2026-08-09 04:46:46 on dist-test-slave-1zqn"
server_key: "69e3b69c3774e6b5fd181d065c48098c"
server_key_iv: "f98dff1de4e65828a4b007f4f6994fce"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:46.885447 20594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "0afbf7315d8c44d586ac52db06e8093f"
format_stamp: "Formatted at 2026-08-09 04:46:46 on dist-test-slave-1zqn"
server_key: "69e3b69c3774e6b5fd181d065c48098c"
server_key_iv: "f98dff1de4e65828a4b007f4f6994fce"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:46.890399 20594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:46:46.906288 20613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:46.907851 20594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:46:46.908008 20594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "0afbf7315d8c44d586ac52db06e8093f"
format_stamp: "Formatted at 2026-08-09 04:46:46 on dist-test-slave-1zqn"
server_key: "69e3b69c3774e6b5fd181d065c48098c"
server_key_iv: "f98dff1de4e65828a4b007f4f6994fce"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:46.908192 20594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:46.943403 20594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:46.946667 20594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:46.947153 20594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:46.956233 20594 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:35357
I20260809 04:46:46.956359 20677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:35357 every 8 connection(s)
I20260809 04:46:46.957561 20594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:46:46.958149 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 20594
I20260809 04:46:46.958344 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:46:46.958710 7852 external_mini_cluster.cc:1428] Setting key 43c99cb61d5ecc9fd732372c766223a6
I20260809 04:46:46.961184 20678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:46.967658 20678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f: Bootstrap starting.
I20260809 04:46:46.969992 20678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:46.970745 20678 log.cc:824] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:46:46 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:46:46.972909 20678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f: No bootstrap required, opened a new log
I20260809 04:46:46.976289 20678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0afbf7315d8c44d586ac52db06e8093f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35357 } }
I20260809 04:46:46.976588 20678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:46.976679 20678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0afbf7315d8c44d586ac52db06e8093f, State: Initialized, Role: FOLLOWER
I20260809 04:46:46.977154 20678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0afbf7315d8c44d586ac52db06e8093f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35357 } }
I20260809 04:46:46.977278 20678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:46:46.977360 20678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:46:46.977450 20678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:46.977638 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:46.960285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.7.171.62:35357)
0809 04:46:46.963909 (+ 3624us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:46.963925 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:46.963972 (+ 47us) server_negotiation.cc:407] Connection header received
0809 04:46:46.964699 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:46.964717 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:46.965131 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:46.965490 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:46.966381 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:46.967590 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:46.968602 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:46.969156 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:46.969326 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:46.972262 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:46.972268 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:46.972283 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:46.972310 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:46.974695 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:46.975202 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:46.975206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:46.975211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:46.975314 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:46.975606 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:46.975608 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:46.975609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:46.975980 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:46.976195 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:46.976472 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 04:46:46.976769 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3474,"thread_start_us":146,"threads_started":1}
I20260809 04:46:46.978561 20678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0afbf7315d8c44d586ac52db06e8093f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35357 } }
I20260809 04:46:46.978933 20678 leader_election.cc:302] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0afbf7315d8c44d586ac52db06e8093f; no voters:
I20260809 04:46:46.979290 20678 leader_election.cc:288] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:46:46.979362 20684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:46:46.979581 20684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [term 1 LEADER]: Becoming Leader. State: Replica: 0afbf7315d8c44d586ac52db06e8093f, State: Running, Role: LEADER
I20260809 04:46:46.980268 20684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0afbf7315d8c44d586ac52db06e8093f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35357 } }
I20260809 04:46:46.980625 20678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:46:46.981472 20684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0afbf7315d8c44d586ac52db06e8093f. Latest consensus state: current_term: 1 leader_uuid: "0afbf7315d8c44d586ac52db06e8093f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0afbf7315d8c44d586ac52db06e8093f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35357 } } }
I20260809 04:46:46.981637 20684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:46.982009 20695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:46:46.981962 20685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0afbf7315d8c44d586ac52db06e8093f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0afbf7315d8c44d586ac52db06e8093f" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35357 } } }
I20260809 04:46:46.982241 20685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:46.985546 20695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:46:46.986040 20678 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 04:46:46.996127 20695 catalog_manager.cc:1347] Generated new cluster ID: 83e1d06246c04c888a194578bcc888ae
I20260809 04:46:46.996301 20695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:46:47.018425 20695 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:46:47.019737 20695 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:46:47.025931 20695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f: Generated new TSK 0
I20260809 04:46:47.026782 20695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:46:47.744709 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:59315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35357
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:47.898672 20732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:47.899017 20732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:47.899091 20732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:47.903525 20732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:47.903589 20732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:47.903651 20732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:46:47.909358 20732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.171.20:59315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:46:47.910543 20732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:47.911777 20732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:47.923295 20743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:47.923447 20740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:47.923491 20732 server_base.cc:1034] running on GCE node
W20260809 04:46:47.923309 20741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:47.924497 20732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:47.925379 20732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:47.926611 20732 hybrid_clock.cc:648] HybridClock initialized: now 1786250807926578 us; error 50 us; skew 500 ppm
Aug 09 04:46:47 dist-test-slave-1zqn krb5kdc[20081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250807, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:47.930831 20732 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:46:47.931993 20732 webserver.cc:460] Webserver started at http://127.7.171.1:37667/ using document root <none> and password file <none>
I20260809 04:46:47.932770 20732 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:47.932860 20732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:47.933106 20732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:46:47 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250807, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:46:47.951771 20732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "2ad5dcd7b4334a748e085c9156eb1796"
format_stamp: "Formatted at 2026-08-09 04:46:47 on dist-test-slave-1zqn"
server_key: "c6eec0564150f4787f6de6160ab867b0"
server_key_iv: "109a20de091b4723b0c8178f4721e12b"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:47.952802 20732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "2ad5dcd7b4334a748e085c9156eb1796"
format_stamp: "Formatted at 2026-08-09 04:46:47 on dist-test-slave-1zqn"
server_key: "c6eec0564150f4787f6de6160ab867b0"
server_key_iv: "109a20de091b4723b0c8178f4721e12b"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:47.958065 20732 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:46:47.976032 20751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:47.977877 20732 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:46:47.978125 20732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "2ad5dcd7b4334a748e085c9156eb1796"
format_stamp: "Formatted at 2026-08-09 04:46:47 on dist-test-slave-1zqn"
server_key: "c6eec0564150f4787f6de6160ab867b0"
server_key_iv: "109a20de091b4723b0c8178f4721e12b"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:47.978403 20732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:48.004926 20732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:48.009338 20732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:48.009665 20732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:48.010532 20732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:48.011723 20732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:48.011824 20732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:48.011920 20732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:48.011974 20732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:48.033629 20732 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:34481
I20260809 04:46:48.035101 20732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:46:48.035617 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 20732
I20260809 04:46:48.035799 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:46:48.036226 7852 external_mini_cluster.cc:1428] Setting key ecc4ea7c6b7ade525547cc3c20924d9a
I20260809 04:46:48.036685 20877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:34481 every 8 connection(s)
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
Aug 09 04:46:48 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250807, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:46:48.063892 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.042169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35357 (local address 127.7.171.1:38931)
0809 04:46:48.045110 (+ 2941us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.045180 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:48.046390 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:48.047572 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:48.047584 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:48.048037 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:48.048823 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.048846 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.050129 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.050133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.051573 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.051585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.052469 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.052474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.052685 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.055454 (+ 2769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:48.055479 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:48.057297 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:48.060038 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:48.060047 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:48.060097 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:48.060501 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:48.060895 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:48.060900 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:48.060902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:48.061108 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:48.061658 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:48.061666 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:48.062066 (+ 400us) client_negotiation.cc:769] Sending connection context
0809 04:46:48.062297 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 04:46:48.062711 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2655,"thread_start_us":160,"threads_started":1}
I20260809 04:46:48.064770 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.043391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:38931 (local address 127.7.171.62:35357)
0809 04:46:48.046936 (+ 3545us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:48.046942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:48.046961 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:46:48.047015 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:48.047018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:48.047075 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:48.047180 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:48.049050 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.049951 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.051732 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.052154 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.052276 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.057547 (+ 5271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:48.057553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:48.057556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:48.057591 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:48.059835 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.060649 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.060654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.060656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.060769 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.061231 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.061235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.061237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.061496 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:48.061598 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:48.064422 (+ 2824us) server_negotiation.cc:299] Negotiation successful
0809 04:46:48.064571 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3431,"thread_start_us":93,"threads_started":1}
I20260809 04:46:48.066855 20878 heartbeater.cc:347] Connected to a master server at 127.7.171.62:35357
I20260809 04:46:48.067159 20878 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:48.067795 20878 heartbeater.cc:511] Master 127.7.171.62:35357 requested a full tablet report, sending...
I20260809 04:46:48.069831 20633 ts_manager.cc:194] Registered new tserver with Master: 2ad5dcd7b4334a748e085c9156eb1796 (127.7.171.1:34481)
I20260809 04:46:48.071408 20633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:38931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:46:48.076021 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:59315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35357
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:46:48 dist-test-slave-1zqn krb5kdc[20081](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 04:46:48.240511 20884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:48.240828 20884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:48.240913 20884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:48.249109 20884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:48.249305 20884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:48.249465 20884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:46:48.258222 20884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.171.20:59315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:46:48.260121 20884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:48.261853 20884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:48.282276 20894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:48.282251 20890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:48.282290 20891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:48.282781 20884 server_base.cc:1034] running on GCE node
I20260809 04:46:48.283464 20884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:48.284370 20884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:48.285631 20884 hybrid_clock.cc:648] HybridClock initialized: now 1786250808285591 us; error 53 us; skew 500 ppm
Aug 09 04:46:48 dist-test-slave-1zqn krb5kdc[20081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:48.289265 20884 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:46:48.290503 20884 webserver.cc:460] Webserver started at http://127.7.171.2:42875/ using document root <none> and password file <none>
I20260809 04:46:48.291237 20884 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:48.291347 20884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:48.291607 20884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:46:48 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:46:48.307579 20884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "a4f5f870b17847bdb9cd4d1148cab817"
format_stamp: "Formatted at 2026-08-09 04:46:48 on dist-test-slave-1zqn"
server_key: "2c38879054890ec9b7882ee0787f2c7a"
server_key_iv: "746f796c7131dfe179eecfccf682f5d2"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:48.308534 20884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "a4f5f870b17847bdb9cd4d1148cab817"
format_stamp: "Formatted at 2026-08-09 04:46:48 on dist-test-slave-1zqn"
server_key: "2c38879054890ec9b7882ee0787f2c7a"
server_key_iv: "746f796c7131dfe179eecfccf682f5d2"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:48.322916 20884 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.001s sys 0.004s
I20260809 04:46:48.340898 20902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:48.342756 20884 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:46:48.342921 20884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "a4f5f870b17847bdb9cd4d1148cab817"
format_stamp: "Formatted at 2026-08-09 04:46:48 on dist-test-slave-1zqn"
server_key: "2c38879054890ec9b7882ee0787f2c7a"
server_key_iv: "746f796c7131dfe179eecfccf682f5d2"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:48.343056 20884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:48.363122 20884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:48.367535 20884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:48.367816 20884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:48.368667 20884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:48.369831 20884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:48.369899 20884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:48.369966 20884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:48.370039 20884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 04:46:48.383464 20884 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:34437
I20260809 04:46:48.384869 20884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:46:48.388240 21027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:34437 every 8 connection(s)
I20260809 04:46:48.394620 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 20884
I20260809 04:46:48.394776 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:46:48.395102 7852 external_mini_cluster.cc:1428] Setting key 0612adba7ea324e39da204ca52550650
Aug 09 04:46:48 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:46:48.407459 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.388961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:58343 (local address 127.7.171.62:35357)
0809 04:46:48.389143 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:48.389148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:48.389170 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:46:48.389929 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:48.389933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:48.389995 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:48.390117 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:48.391614 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.392546 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.396178 (+ 3632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.396551 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.396665 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.402513 (+ 5848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:48.402517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:48.402520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:48.402553 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:48.404682 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.405410 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.405415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.405417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.405533 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.405988 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.405992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.405993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.406238 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:48.406321 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:48.407124 (+ 803us) server_negotiation.cc:299] Negotiation successful
0809 04:46:48.407280 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:46:48.409686 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.387859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35357 (local address 127.7.171.2:58343)
0809 04:46:48.388592 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.388644 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:48.389673 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:48.390352 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:48.390362 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:48.390751 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:48.391430 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.391451 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.393502 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.393506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.394946 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.394956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.399433 (+ 4477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.399438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.399668 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.400494 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:48.400512 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:48.402288 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:48.404853 (+ 2565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:48.404861 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:48.404881 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:48.405273 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:48.405657 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:48.405660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:48.405662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:48.405872 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:48.406406 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:48.406415 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:48.406814 (+ 399us) client_negotiation.cc:769] Sending connection context
0809 04:46:48.407932 (+ 1118us) client_negotiation.cc:240] Negotiation successful
0809 04:46:48.408419 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":204,"threads_started":1}
I20260809 04:46:48.411337 21028 heartbeater.cc:347] Connected to a master server at 127.7.171.62:35357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
I20260809 04:46:48.411645 21028 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:48.412379 21028 heartbeater.cc:511] Master 127.7.171.62:35357 requested a full tablet report, sending...
I20260809 04:46:48.413744 20633 ts_manager.cc:194] Registered new tserver with Master: a4f5f870b17847bdb9cd4d1148cab817 (127.7.171.2:34437)
I20260809 04:46:48.414443 20633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:58343
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:46:48.437779 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:59315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35357
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:46:48 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
W20260809 04:46:48.618204 21033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:48.618705 21033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:48.618825 21033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:48.625052 21033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:48.625156 21033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:48.625252 21033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:46:48.633685 21033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.171.20:59315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:46:48.635210 21033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:48.636641 21033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:48.649616 21039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:48.649814 21042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:48.649616 21040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:48.651162 21033 server_base.cc:1034] running on GCE node
I20260809 04:46:48.651719 21033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:48.652556 21033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:48.653744 21033 hybrid_clock.cc:648] HybridClock initialized: now 1786250808653717 us; error 48 us; skew 500 ppm
Aug 09 04:46:48 dist-test-slave-1zqn krb5kdc[20081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:48.657558 21033 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:46:48.658747 21033 webserver.cc:460] Webserver started at http://127.7.171.3:45697/ using document root <none> and password file <none>
I20260809 04:46:48.659518 21033 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:48.659615 21033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:48.659929 21033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:46:48 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:46:48.681777 21033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "f4e4af9798684b41b2caa62a7747b7b9"
format_stamp: "Formatted at 2026-08-09 04:46:48 on dist-test-slave-1zqn"
server_key: "4fabad4cf8505dad1f92bbb9bdcf2faf"
server_key_iv: "e8c6c532a700a8b0cf56c76ac23b532f"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:48.682658 21033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "f4e4af9798684b41b2caa62a7747b7b9"
format_stamp: "Formatted at 2026-08-09 04:46:48 on dist-test-slave-1zqn"
server_key: "4fabad4cf8505dad1f92bbb9bdcf2faf"
server_key_iv: "e8c6c532a700a8b0cf56c76ac23b532f"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:48.688548 21033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:46:48.711670 21049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:48.713796 21033 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260809 04:46:48.713940 21033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "f4e4af9798684b41b2caa62a7747b7b9"
format_stamp: "Formatted at 2026-08-09 04:46:48 on dist-test-slave-1zqn"
server_key: "4fabad4cf8505dad1f92bbb9bdcf2faf"
server_key_iv: "e8c6c532a700a8b0cf56c76ac23b532f"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:48.714083 21033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:48.740929 21033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:48.744784 21033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:48.745029 21033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:48.745774 21033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:48.746906 21033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:48.747001 21033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:48.747085 21033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:48.747126 21033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:48.758392 21033 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:45005
I20260809 04:46:48.759649 21033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:46:48.769528 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 21033
I20260809 04:46:48.769701 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:46:48.770047 7852 external_mini_cluster.cc:1428] Setting key 65818766d27a778735b8919397e50585
I20260809 04:46:48.778149 21171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:45005 every 8 connection(s)
Aug 09 04:46:48 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:46:48.808040 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.785618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:39931 (local address 127.7.171.62:35357)
0809 04:46:48.785960 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:48.785965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:48.785989 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:46:48.786840 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:48.786844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:48.786909 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:48.787004 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:48.791067 (+ 4063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.791968 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.794214 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.794535 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.794618 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.802823 (+ 8205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:48.802827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:48.802830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:48.802868 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:48.804716 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.805540 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.805544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.805546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.805646 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.806217 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.806221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.806223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.806532 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:48.806637 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:48.807660 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0809 04:46:48.807827 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260809 04:46:48.810595 21174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.783800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35357 (local address 127.7.171.3:39931)
0809 04:46:48.784782 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.784847 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:48.786353 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:48.787246 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:48.787257 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:48.787791 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:48.790840 (+ 3049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.790872 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.792216 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.792222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.794062 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.794076 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.794841 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.794845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.795110 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.800285 (+ 5175us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:48.800308 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:48.802599 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:48.804894 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:48.804901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:48.804922 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:48.805405 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:48.805799 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:48.805802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:48.805804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:48.806072 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:48.806756 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:48.806764 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:48.807282 (+ 518us) client_negotiation.cc:769] Sending connection context
0809 04:46:48.808770 (+ 1488us) client_negotiation.cc:240] Negotiation successful
0809 04:46:48.809232 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":336,"threads_started":1}
I20260809 04:46:48.812501 21172 heartbeater.cc:347] Connected to a master server at 127.7.171.62:35357
I20260809 04:46:48.812763 21172 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:48.813338 21172 heartbeater.cc:511] Master 127.7.171.62:35357 requested a full tablet report, sending...
I20260809 04:46:48.814591 20633 ts_manager.cc:194] Registered new tserver with Master: f4e4af9798684b41b2caa62a7747b7b9 (127.7.171.3:45005)
I20260809 04:46:48.815434 20633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:39931
I20260809 04:46:48.824992 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:46:48.842484 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.828544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.7.171.62:35357)
0809 04:46:48.828695 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:48.828700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:48.832486 (+ 3786us) server_negotiation.cc:407] Connection header received
0809 04:46:48.832719 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:48.832723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:48.832780 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:48.832842 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:48.834056 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.834831 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.836017 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.836370 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.836445 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.838484 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:48.838489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:48.838492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:48.838532 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:48.840494 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.841048 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.841052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.841054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.841141 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.841437 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.841441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.841443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.841672 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:48.841780 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:48.842191 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 04:46:48.842322 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:46:48.846812 20633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:46:48.855855 20633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:46:48.880573 21185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.872655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45005 (local address 127.0.0.1:59954)
0809 04:46:48.873675 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.873728 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:48.873854 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:48.874715 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:48.874722 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:48.874748 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:48.874914 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.874922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.876897 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.876901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.878604 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.878615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.879927 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.879931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.880106 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.880166 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 04:46:48.880267 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 04:46:48.880411 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":165,"threads_started":1}
I20260809 04:46:48.881386 21187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.873021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.7.171.1:34481)
0809 04:46:48.873420 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:48.873428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:48.875054 (+ 1626us) server_negotiation.cc:407] Connection header received
0809 04:46:48.875284 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:48.875289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:48.875481 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:48.875593 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:48.876007 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.877655 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.879223 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.880184 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.880286 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.880430 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:48.881108 (+ 678us) server_negotiation.cc:299] Negotiation successful
0809 04:46:48.881209 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":177,"threads_started":1}
I20260809 04:46:48.881704 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.873643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59954 (local address 127.7.171.3:45005)
0809 04:46:48.874112 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:48.874136 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:48.874165 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:46:48.874251 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:48.874257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:48.874484 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:48.875090 (+ 606us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:48.875237 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.876761 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.878889 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.879772 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.881118 (+ 1346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.881313 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:48.881366 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 04:46:48.881503 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":116,"threads_started":1}
I20260809 04:46:48.882262 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.872663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:34481 (local address 127.0.0.1:46656)
0809 04:46:48.874901 (+ 2238us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.874934 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:48.875128 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:48.875761 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:48.875764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:48.875781 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:48.875865 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.875871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.877804 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.877808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.879079 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.879090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.880549 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.880552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.880642 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.880964 (+ 322us) client_negotiation.cc:769] Sending connection context
0809 04:46:48.881862 (+ 898us) client_negotiation.cc:240] Negotiation successful
0809 04:46:48.882127 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1646,"mutex_wait_us":3,"thread_start_us":1497,"threads_started":1}
I20260809 04:46:48.885284 21102 tablet_service.cc:1467] Processing CreateTablet for tablet 6f82cb73913a472cba4b102e3eac96fa (DEFAULT_TABLE table=test-table [id=7405c86681b84f059618b0bb41c57153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:48.885758 20806 tablet_service.cc:1467] Processing CreateTablet for tablet 6f82cb73913a472cba4b102e3eac96fa (DEFAULT_TABLE table=test-table [id=7405c86681b84f059618b0bb41c57153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:48.886806 21102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f82cb73913a472cba4b102e3eac96fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:48.886893 20806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f82cb73913a472cba4b102e3eac96fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:48.897267 21185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.874436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:34437 (local address 127.0.0.1:52910)
0809 04:46:48.880886 (+ 6450us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.880902 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:48.880984 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:48.892163 (+ 11179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:48.892167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:48.892195 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:48.892311 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.892319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.894157 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.894161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.895318 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:48.895331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.896810 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.896814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:48.896911 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.896938 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:46:48.897019 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 04:46:48.897102 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6260,"thread_start_us":150,"threads_started":1}
I20260809 04:46:48.897866 21192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.874587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52910 (local address 127.7.171.2:34437)
0809 04:46:48.883850 (+ 9263us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:48.883907 (+ 57us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:48.883936 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:46:48.884016 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:48.884021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:48.884258 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:48.884458 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:48.892505 (+ 8047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.893960 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.895571 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.896452 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.896564 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.896737 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:48.897554 (+ 817us) server_negotiation.cc:299] Negotiation successful
0809 04:46:48.897682 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9145,"thread_start_us":136,"threads_started":1}
I20260809 04:46:48.901501 20955 tablet_service.cc:1467] Processing CreateTablet for tablet 6f82cb73913a472cba4b102e3eac96fa (DEFAULT_TABLE table=test-table [id=7405c86681b84f059618b0bb41c57153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:48.902719 20955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f82cb73913a472cba4b102e3eac96fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:48.903648 21194 tablet_bootstrap.cc:492] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: Bootstrap starting.
I20260809 04:46:48.907166 21194 tablet_bootstrap.cc:654] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:48.908676 21194 log.cc:824] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:48.909430 21193 tablet_bootstrap.cc:492] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796: Bootstrap starting.
I20260809 04:46:48.911190 21194 tablet_bootstrap.cc:492] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: No bootstrap required, opened a new log
I20260809 04:46:48.911418 21194 ts_tablet_manager.cc:1397] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 04:46:48.912341 21193 tablet_bootstrap.cc:654] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:48.913225 21193 log.cc:824] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:48.918079 21193 tablet_bootstrap.cc:492] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796: No bootstrap required, opened a new log
I20260809 04:46:48.918351 21193 ts_tablet_manager.cc:1397] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 04:46:48.919988 21194 raft_consensus.cc:348] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad5dcd7b4334a748e085c9156eb1796" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34481 } } peers { permanent_uuid: "f4e4af9798684b41b2caa62a7747b7b9" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45005 } } peers { permanent_uuid: "a4f5f870b17847bdb9cd4d1148cab817" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34437 } }
I20260809 04:46:48.920439 21194 raft_consensus.cc:374] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:48.920568 21194 raft_consensus.cc:729] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e4af9798684b41b2caa62a7747b7b9, State: Initialized, Role: FOLLOWER
I20260809 04:46:48.921376 21194 consensus_queue.cc:260] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad5dcd7b4334a748e085c9156eb1796" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34481 } } peers { permanent_uuid: "f4e4af9798684b41b2caa62a7747b7b9" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45005 } } peers { permanent_uuid: "a4f5f870b17847bdb9cd4d1148cab817" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34437 } }
I20260809 04:46:48.922781 21194 ts_tablet_manager.cc:1428] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260809 04:46:48.923264 21172 heartbeater.cc:503] Master 127.7.171.62:35357 was elected leader, sending a full tablet report...
I20260809 04:46:48.924947 21193 raft_consensus.cc:348] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad5dcd7b4334a748e085c9156eb1796" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34481 } } peers { permanent_uuid: "f4e4af9798684b41b2caa62a7747b7b9" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45005 } } peers { permanent_uuid: "a4f5f870b17847bdb9cd4d1148cab817" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34437 } }
I20260809 04:46:48.925841 21196 tablet_bootstrap.cc:492] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: Bootstrap starting.
I20260809 04:46:48.928136 21193 raft_consensus.cc:374] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:48.928269 21193 raft_consensus.cc:729] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad5dcd7b4334a748e085c9156eb1796, State: Initialized, Role: FOLLOWER
I20260809 04:46:48.928871 21193 consensus_queue.cc:260] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad5dcd7b4334a748e085c9156eb1796" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34481 } } peers { permanent_uuid: "f4e4af9798684b41b2caa62a7747b7b9" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45005 } } peers { permanent_uuid: "a4f5f870b17847bdb9cd4d1148cab817" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34437 } }
I20260809 04:46:48.930286 21193 ts_tablet_manager.cc:1428] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796: Time spent starting tablet: real 0.012s user 0.003s sys 0.005s
I20260809 04:46:48.931242 21196 tablet_bootstrap.cc:654] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:48.931744 20878 heartbeater.cc:503] Master 127.7.171.62:35357 was elected leader, sending a full tablet report...
I20260809 04:46:48.933223 21196 log.cc:824] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:48.944550 21196 tablet_bootstrap.cc:492] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: No bootstrap required, opened a new log
I20260809 04:46:48.944875 21196 ts_tablet_manager.cc:1397] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.000s
I20260809 04:46:48.951143 21196 raft_consensus.cc:348] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad5dcd7b4334a748e085c9156eb1796" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34481 } } peers { permanent_uuid: "f4e4af9798684b41b2caa62a7747b7b9" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45005 } } peers { permanent_uuid: "a4f5f870b17847bdb9cd4d1148cab817" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34437 } }
I20260809 04:46:48.951545 21196 raft_consensus.cc:374] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:48.951663 21196 raft_consensus.cc:729] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f5f870b17847bdb9cd4d1148cab817, State: Initialized, Role: FOLLOWER
I20260809 04:46:48.952291 21196 consensus_queue.cc:260] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad5dcd7b4334a748e085c9156eb1796" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34481 } } peers { permanent_uuid: "f4e4af9798684b41b2caa62a7747b7b9" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45005 } } peers { permanent_uuid: "a4f5f870b17847bdb9cd4d1148cab817" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34437 } }
I20260809 04:46:48.953394 21028 heartbeater.cc:503] Master 127.7.171.62:35357 was elected leader, sending a full tablet report...
I20260809 04:46:48.953711 21196 ts_tablet_manager.cc:1428] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
Aug 09 04:46:48 dist-test-slave-1zqn krb5kdc[20081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250808, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
W20260809 04:46:49.033356 21173 tablet.cc:2369] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:46:49.041327 20879 tablet.cc:2369] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:46:49.087828 21201 raft_consensus.cc:482] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:46:49.088271 21201 raft_consensus.cc:504] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad5dcd7b4334a748e085c9156eb1796" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34481 } } peers { permanent_uuid: "f4e4af9798684b41b2caa62a7747b7b9" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45005 } } peers { permanent_uuid: "a4f5f870b17847bdb9cd4d1148cab817" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34437 } }
I20260809 04:46:49.089993 21201 leader_election.cc:288] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ad5dcd7b4334a748e085c9156eb1796 (127.7.171.1:34481), f4e4af9798684b41b2caa62a7747b7b9 (127.7.171.3:45005)
I20260809 04:46:49.104977 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:49.091607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:49105 (local address 127.7.171.3:45005)
0809 04:46:49.091750 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:49.091754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:49.098758 (+ 7004us) server_negotiation.cc:407] Connection header received
0809 04:46:49.098905 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:49.098909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:49.098963 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:49.099041 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:49.100672 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:49.101377 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:49.103429 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:49.104106 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:49.104193 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:49.104269 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:49.104731 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 04:46:49.104820 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:46:49.105784 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:49.091081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45005 (local address 127.7.171.2:49105)
0809 04:46:49.098588 (+ 7507us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:49.098609 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:49.098798 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:49.100219 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:49.100223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:49.100260 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:49.100532 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:49.100540 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:49.101635 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:49.101639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:49.103166 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:49.103177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:49.104373 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:49.104377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:49.104567 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:49.104593 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:46:49.105476 (+ 883us) client_negotiation.cc:240] Negotiation successful
0809 04:46:49.105560 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7423,"thread_start_us":164,"threads_started":1}
I20260809 04:46:49.106906 21112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f82cb73913a472cba4b102e3eac96fa" candidate_uuid: "a4f5f870b17847bdb9cd4d1148cab817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e4af9798684b41b2caa62a7747b7b9" is_pre_election: true
I20260809 04:46:49.107362 21112 raft_consensus.cc:2454] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4f5f870b17847bdb9cd4d1148cab817 in term 0.
I20260809 04:46:49.108223 20905 leader_election.cc:302] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4f5f870b17847bdb9cd4d1148cab817, f4e4af9798684b41b2caa62a7747b7b9; no voters:
I20260809 04:46:49.108626 21201 raft_consensus.cc:2781] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:46:49.108709 21201 raft_consensus.cc:482] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:46:49.108750 21201 raft_consensus.cc:3037] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:49.109921 21187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:49.090584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:37029 (local address 127.7.171.1:34481)
0809 04:46:49.090772 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:49.090778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:49.096622 (+ 5844us) server_negotiation.cc:407] Connection header received
0809 04:46:49.096857 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:49.096861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:49.096914 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:49.096964 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:49.097533 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:49.098353 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:49.108595 (+ 10242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:49.109217 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:49.109301 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:49.109365 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:49.109719 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 04:46:49.109798 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:46:49.110347 21204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:49.090465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:34481 (local address 127.7.171.2:37029)
0809 04:46:49.096471 (+ 6006us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:49.096489 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:49.096723 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:49.097116 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:49.097119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:49.097226 (+ 107us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:49.097383 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:49.097390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:49.098516 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:49.098520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:49.108418 (+ 9898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:49.108431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:49.109478 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:49.109481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:49.109581 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:49.109603 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:46:49.110124 (+ 521us) client_negotiation.cc:240] Negotiation successful
0809 04:46:49.110207 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5906,"thread_start_us":222,"threads_started":1}
I20260809 04:46:49.110843 21201 raft_consensus.cc:504] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad5dcd7b4334a748e085c9156eb1796" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34481 } } peers { permanent_uuid: "f4e4af9798684b41b2caa62a7747b7b9" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45005 } } peers { permanent_uuid: "a4f5f870b17847bdb9cd4d1148cab817" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34437 } }
I20260809 04:46:49.110944 20809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f82cb73913a472cba4b102e3eac96fa" candidate_uuid: "a4f5f870b17847bdb9cd4d1148cab817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ad5dcd7b4334a748e085c9156eb1796" is_pre_election: true
I20260809 04:46:49.111258 20809 raft_consensus.cc:2454] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4f5f870b17847bdb9cd4d1148cab817 in term 0.
I20260809 04:46:49.111279 21201 leader_election.cc:288] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [CANDIDATE]: Term 1 election: Requested vote from peers 2ad5dcd7b4334a748e085c9156eb1796 (127.7.171.1:34481), f4e4af9798684b41b2caa62a7747b7b9 (127.7.171.3:45005)
I20260809 04:46:49.112257 20809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f82cb73913a472cba4b102e3eac96fa" candidate_uuid: "a4f5f870b17847bdb9cd4d1148cab817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ad5dcd7b4334a748e085c9156eb1796"
I20260809 04:46:49.112368 20809 raft_consensus.cc:3037] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:49.114746 20809 raft_consensus.cc:2454] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4f5f870b17847bdb9cd4d1148cab817 in term 1.
I20260809 04:46:49.115085 20904 leader_election.cc:302] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ad5dcd7b4334a748e085c9156eb1796, a4f5f870b17847bdb9cd4d1148cab817; no voters:
I20260809 04:46:49.115862 21112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f82cb73913a472cba4b102e3eac96fa" candidate_uuid: "a4f5f870b17847bdb9cd4d1148cab817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e4af9798684b41b2caa62a7747b7b9"
I20260809 04:46:49.116003 21112 raft_consensus.cc:3037] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:49.117476 21112 raft_consensus.cc:2454] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4f5f870b17847bdb9cd4d1148cab817 in term 1.
I20260809 04:46:49.118065 21201 raft_consensus.cc:2781] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:46:49.118294 21201 raft_consensus.cc:686] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 1 LEADER]: Becoming Leader. State: Replica: a4f5f870b17847bdb9cd4d1148cab817, State: Running, Role: LEADER
I20260809 04:46:49.118764 21201 consensus_queue.cc:237] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad5dcd7b4334a748e085c9156eb1796" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34481 } } peers { permanent_uuid: "f4e4af9798684b41b2caa62a7747b7b9" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45005 } } peers { permanent_uuid: "a4f5f870b17847bdb9cd4d1148cab817" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34437 } }
I20260809 04:46:49.123250 20632 catalog_manager.cc:5495] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4f5f870b17847bdb9cd4d1148cab817 (127.7.171.2). New cstate: current_term: 1 leader_uuid: "a4f5f870b17847bdb9cd4d1148cab817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ad5dcd7b4334a748e085c9156eb1796" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e4af9798684b41b2caa62a7747b7b9" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4f5f870b17847bdb9cd4d1148cab817" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34437 } health_report { overall_health: HEALTHY } } }
W20260809 04:46:49.138293 21029 tablet.cc:2369] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:46:49.184885 21192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:49.179320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52918 (local address 127.7.171.2:34437)
0809 04:46:49.179490 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:49.179495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:49.179512 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:46:49.179606 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:49.179609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:49.179679 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:49.179796 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:46:49.180250 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:49.181098 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:49.182480 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:49.182829 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:49.182926 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:49.183353 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:46:49.183933 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:46:49.184047 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:49.184634 (+ 587us) server_negotiation.cc:299] Negotiation successful
0809 04:46:49.184741 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:46:49.192853 21112 raft_consensus.cc:1261] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9 [term 1 FOLLOWER]: Refusing update from remote peer a4f5f870b17847bdb9cd4d1148cab817: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:46:49.193779 21206 consensus_queue.cc:1035] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e4af9798684b41b2caa62a7747b7b9" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:46:49.197490 20809 raft_consensus.cc:1261] T 6f82cb73913a472cba4b102e3eac96fa P 2ad5dcd7b4334a748e085c9156eb1796 [term 1 FOLLOWER]: Refusing update from remote peer a4f5f870b17847bdb9cd4d1148cab817: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:46:49.198261 21206 consensus_queue.cc:1035] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ad5dcd7b4334a748e085c9156eb1796" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:46:49.230610 21212 mvcc.cc:205] Tried to move back new op lower bound from 7316483314443476992 to 7316483314155040768. Current Snapshot: MvccSnapshot[applied={T|T < 7316483314443476992}]
I20260809 04:46:49.230885 21214 mvcc.cc:205] Tried to move back new op lower bound from 7316483314443476992 to 7316483314155040768. Current Snapshot: MvccSnapshot[applied={T|T < 7316483314443476992}]
I20260809 04:46:49.241884 20633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:46:49.242105 20633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:46:49.245323 20633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0afbf7315d8c44d586ac52db06e8093f: Sending DeleteTablet for 3 replicas of tablet 6f82cb73913a472cba4b102e3eac96fa
I20260809 04:46:49.246771 21102 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f82cb73913a472cba4b102e3eac96fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:49 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:59954
I20260809 04:46:49.246825 20806 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f82cb73913a472cba4b102e3eac96fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:49 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:46656
I20260809 04:46:49.247566 20955 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f82cb73913a472cba4b102e3eac96fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:49 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:52910
I20260809 04:46:49.248739 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 20732
I20260809 04:46:49.254395 21224 tablet_replica.cc:330] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: stopping tablet replica
I20260809 04:46:49.254797 21224 raft_consensus.cc:2229] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:46:49.255298 21224 raft_consensus.cc:2258] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:46:49.261150 21224 ts_tablet_manager.cc:1905] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:46:49.262234 21222 tablet_replica.cc:330] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: stopping tablet replica
I20260809 04:46:49.262634 21222 raft_consensus.cc:2229] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:46:49.263051 21222 raft_consensus.cc:2258] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:46:49.264397 21224 ts_tablet_manager.cc:1918] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:46:49.264513 21224 log.cc:1196] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/6f82cb73913a472cba4b102e3eac96fa
I20260809 04:46:49.264866 21224 ts_tablet_manager.cc:1939] T 6f82cb73913a472cba4b102e3eac96fa P a4f5f870b17847bdb9cd4d1148cab817: Deleting consensus metadata
I20260809 04:46:49.264865 21222 ts_tablet_manager.cc:1905] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:46:49.266253 20616 catalog_manager.cc:4841] TS a4f5f870b17847bdb9cd4d1148cab817 (127.7.171.2:34437): tablet 6f82cb73913a472cba4b102e3eac96fa (table test-table [id=7405c86681b84f059618b0bb41c57153]) successfully deleted
I20260809 04:46:49.267624 21222 ts_tablet_manager.cc:1918] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:46:49.267745 21222 log.cc:1196] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/6f82cb73913a472cba4b102e3eac96fa
I20260809 04:46:49.268229 21222 ts_tablet_manager.cc:1939] T 6f82cb73913a472cba4b102e3eac96fa P f4e4af9798684b41b2caa62a7747b7b9: Deleting consensus metadata
W20260809 04:46:49.269198 20616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:34481 (error 108)
I20260809 04:46:49.269565 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 20884
I20260809 04:46:49.271826 20617 catalog_manager.cc:4841] TS f4e4af9798684b41b2caa62a7747b7b9 (127.7.171.3:45005): tablet 6f82cb73913a472cba4b102e3eac96fa (table test-table [id=7405c86681b84f059618b0bb41c57153]) successfully deleted
I20260809 04:46:49.272307 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:49.271844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:34481 (local address 127.0.0.1:46660)
0809 04:46:49.272077 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:49.272179 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:34481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260809 04:46:49.276913 20616 catalog_manager.cc:4584] TS 2ad5dcd7b4334a748e085c9156eb1796 (127.7.171.1:34481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f82cb73913a472cba4b102e3eac96fa: Network error: Client connection negotiation failed: client connection to 127.7.171.1:34481: connect: Connection refused (error 111)
I20260809 04:46:49.279349 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 21033
I20260809 04:46:49.288512 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 20594
I20260809 04:46:49.300576 7852 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 04:46:49.645614 7852 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 04:46:49.645795 7852 mini_ranger.cc:66] Stopping Ranger...
I20260809 04:46:49.990962 7852 mini_ranger.cc:68] Stopped Ranger
2026-08-09 04:46:49.991 UTC [20118] LOG: received smart shutdown request
2026-08-09 04:46:49.995 UTC [20118] LOG: background worker "logical replication launcher" (PID 20128) exited with exit code 1
2026-08-09 04:46:49.996 UTC [20122] LOG: shutting down
2026-08-09 04:46:50.177 UTC [20118] LOG: database system is shut down
2026-08-09T04:46:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49035 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:46:50 dist-test-slave-1zqn krb5kdc[21239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:46:50 dist-test-slave-1zqn krb5kdc[21239](info): set up 2 sockets
Aug 09 04:46:50 dist-test-slave-1zqn krb5kdc[21239](info): commencing operation
krb5kdc: starting...
W20260809 04:46:51.461474 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:46:51 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:46:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:46:51Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.171.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.171.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/rangeradmin_127.7.171.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/rangeradmin_127.7.171.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.171.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.171.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/rangerlookup_127.7.171.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/rangerlookup_127.7.171.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.171.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.171.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/HTTP_127.7.171.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/HTTP_127.7.171.20@KRBTEST.COM.keytab.
I20260809 04:46:51.655683 7852 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfTSqH0/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/postgres -l logfile start
2026-08-09 04:46:54.336 UTC [21270] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 04:46:54.336 UTC [21270] LOG: listening on IPv4 address "127.7.171.20", port 45829
2026-08-09 04:46:54.340 UTC [21270] LOG: listening on Unix socket "/tmp/.s.PGSQL.45829"
2026-08-09 04:46:54.362 UTC [21274] LOG: database system was shut down at 2026-08-09 04:46:52 UTC
2026-08-09 04:46:54.366 UTC [21270] LOG: database system is ready to accept connections
I20260809 04:46:55.371320 7852 mini_postgres.cc:96] Postgres bound to 45829
2026-08-09 04:46:55.379 UTC [21282] FATAL: database "slave" does not exist
127.7.171.20:45829 - accepting connections
I20260809 04:46:55.379812 7852 mini_ranger.cc:161] Starting Ranger...
I20260809 04:46:55.396416 7852 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 04:46:55.942065 7852 mini_ranger.cc:87] Created ranger Postgres database
I20260809 04:46:55.942173 7852 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-admin
2026-08-09 04:46:56,241 [I] DB FLAVOR :POSTGRES
2026-08-09 04:46:56,242 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:46:56,242 [I] Checking connection..
2026-08-09 04:46:56,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 04:46:56,521 [I] Checking connection passed.
2026-08-09 04:46:56,522 [I] --------- Verifying version history table ---------
2026-08-09 04:46:56,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:46:56,776 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:46:56,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:46:57,033 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:46:57,034 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 04:46:57,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 04:46:57.318 UTC [21354] WARNING: there is no transaction in progress
2026-08-09 04:46:57,326 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 04:46:57,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:46:57,600 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 04:46:57,600 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:46:57,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 04:46:57,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 04:46:58,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1zqn.c.gcp-upstream.internal','N') ;"
2026-08-09 04:46:58,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:46:58,654 [I] Table x_portal_user does not exist in database ranger
2026-08-09 04:46:58,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:46:58,909 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 04:46:58,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:46:59,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:46:59,449 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 04:46:59,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 04:47:00.462 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.478 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.495 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.511 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.530 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.551 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.610 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.618 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.634 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.645 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.656 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.672 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.681 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.687 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.694 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:00.701 UTC [21470] WARNING: there is no transaction in progress
2026-08-09 04:47:01,390 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 04:47:01,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 04:47:01,646 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 04:47:01,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:47:01,929 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 04:47:01,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:47:02,207 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 04:47:02,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:47:02,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:47:02,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1zqn.c.gcp-upstream.internal';"
2026-08-09 04:47:02,998 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 04:47:02,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 04:47:03,251 [I] DB_PATCHES have already been applied
I20260809 04:47:03.257236 7852 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfTSqH0/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 04:47:03.257308 7852 mini_ranger.cc:193] Using host: 127.7.171.20
I20260809 04:47:03.261776 7852 mini_ranger.cc:227] Ranger admin URL: http://127.7.171.20:41535
Aug 09, 2026 4:47:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:47:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:47:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:47:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:47:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:47:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:47:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 04:47:04.476001 7852 mini_ranger.cc:160] Time spent starting Ranger: real 9.096s user 0.005s sys 0.008s
Aug 09, 2026 4:47:25 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:47:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 4:47:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:47:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:47:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 04:47:27.110072 7852 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.171.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.171.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/rangerkms_127.7.171.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.171.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/rangerkms_127.7.171.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.171.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.171.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.171.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/HTTP_127.7.171.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.171.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/HTTP_127.7.171.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 04:47:27 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250847, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 04:47:27.202682 7852 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 04:47:27.213878 7852 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 04:47:28.145396 7852 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 04:47:28.145496 7852 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-kms
I20260809 04:47:28.145519 7852 mini_ranger_kms.cc:227] Using postgres at 127.7.171.20:45829
2026-08-09 04:47:28,334 [I] DB FLAVOR :POSTGRES
2026-08-09 04:47:28,334 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:47:28,334 [I] Checking connection
2026-08-09 04:47:28,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 04:47:28,584 [I] connection success
2026-08-09 04:47:28,585 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 04:47:28,585 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 04:47:28,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 04:47:28,832 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 04:47:28,833 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:47:28,833 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 04:47:28,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.171.20:45829/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 04:47:29,112 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 04:47:29.603391 7852 mini_ranger_kms.cc:310] Created kms service
I20260809 04:47:29.861917 7852 mini_ranger_kms.cc:326] Created kudu user
I20260809 04:47:29.907855 7852 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 04:47:29 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250847, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:47:30.341184 7852 mini_ranger_kms.cc:384] Added ranger policy
I20260809 04:47:30.341434 7852 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfTSqH0/build/debug/bin/postgresql.jar:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfTSqH0/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 04:47:30.341459 7852 mini_ranger_kms.cc:240] Using host: 127.7.171.20
I20260809 04:47:30.345997 7852 mini_ranger_kms.cc:276] Ranger KMS PID: 21664
I20260809 04:47:30.346087 7852 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.171.20:38285
Aug 09, 2026 4:47:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:47:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:47:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:47:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:47:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:47:31 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 04:47:32 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.171.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:32.777788 7852 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.575s user 0.008s sys 0.165s
I20260809 04:47:32.777935 7852 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 04:47:32.778013 7852 mini_ranger_kms.cc:401] 127.7.171.20:38285/kms/v1/keys
Aug 09 04:47:32 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.171.20@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
Aug 09 04:47:33 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.171.20@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
Aug 09 04:47:35 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:47:35.755343 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43957
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:43957
--ranger_config_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:38285/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:35.919060 21715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:35.919404 21715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:35.919482 21715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:35.923784 21715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:35.923871 21715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:35.923915 21715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:35.923974 21715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:35.924012 21715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:35.929824 21715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.171.20:38285/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:43957
--ranger_config_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:35.930920 21715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:35.931994 21715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:35.941802 21723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:35.941792 21721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:35.941787 21720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:35.941847 21715 server_base.cc:1034] running on GCE node
I20260809 04:47:35.942730 21715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:35.943830 21715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:35.945024 21715 hybrid_clock.cc:648] HybridClock initialized: now 1786250855944994 us; error 48 us; skew 500 ppm
Aug 09 04:47:35 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250855, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:35.948304 21715 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:47:35.949155 21715 webserver.cc:460] Webserver started at http://127.7.171.62:42757/ using document root <none> and password file <none>
I20260809 04:47:35.949697 21715 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:35.949777 21715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:35.949995 21715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:47:35 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250855, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:47:36.004004 21715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "f30f2efa53334d89aceeee88f6956f90"
format_stamp: "Formatted at 2026-08-09 04:47:35 on dist-test-slave-1zqn"
server_key: "935c48915646f468b3fd5cb2a7666cbb"
server_key_iv: "f1d74c296d880bdc04f935360735565c"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:36.005000 21715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "f30f2efa53334d89aceeee88f6956f90"
format_stamp: "Formatted at 2026-08-09 04:47:35 on dist-test-slave-1zqn"
server_key: "935c48915646f468b3fd5cb2a7666cbb"
server_key_iv: "f1d74c296d880bdc04f935360735565c"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:36.009747 21715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:47:36.029865 21731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:36.031387 21715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 04:47:36.031538 21715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "f30f2efa53334d89aceeee88f6956f90"
format_stamp: "Formatted at 2026-08-09 04:47:35 on dist-test-slave-1zqn"
server_key: "935c48915646f468b3fd5cb2a7666cbb"
server_key_iv: "f1d74c296d880bdc04f935360735565c"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:36.031692 21715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:36.061473 21715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:36.064990 21715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:36.065291 21715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:36.082331 21715 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:43957
I20260809 04:47:36.082553 21787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:43957 every 8 connection(s)
I20260809 04:47:36.083580 21715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:47:36.086478 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 21715
I20260809 04:47:36.086611 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:47:36.086966 7852 external_mini_cluster.cc:1428] Setting key b97662bb7c6cde4299d776988d4c4691
I20260809 04:47:36.087021 21788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:36.097200 21788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90: Bootstrap starting.
I20260809 04:47:36.100035 21788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90: Neither blocks nor log segments found. Creating new log.
Aug 09 04:47:36 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:36.101507 21788 log.cc:824] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:36.103863 21788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90: No bootstrap required, opened a new log
I20260809 04:47:36.108536 21794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:36.088709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52252 (local address 127.7.171.62:43957)
0809 04:47:36.089358 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:36.089425 (+ 67us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:36.089473 (+ 48us) server_negotiation.cc:407] Connection header received
0809 04:47:36.091048 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:36.091076 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:36.091592 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:36.092035 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:36.093130 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:36.095124 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:36.096615 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:36.097387 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:36.097598 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:36.101359 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:36.101367 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:36.101410 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:36.101459 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:36.104562 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:36.105169 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:36.105176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:36.105183 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:36.105378 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:36.105726 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:36.105730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:36.105732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:36.106429 (+ 697us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:36.106789 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:36.107034 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 04:47:36.107346 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":151,"threads_started":1}
I20260809 04:47:36.108606 21788 raft_consensus.cc:348] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } }
I20260809 04:47:36.109007 21788 raft_consensus.cc:374] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:36.109095 21788 raft_consensus.cc:729] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30f2efa53334d89aceeee88f6956f90, State: Initialized, Role: FOLLOWER
I20260809 04:47:36.109627 21788 consensus_queue.cc:260] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } }
I20260809 04:47:36.109786 21788 raft_consensus.cc:388] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:47:36.109867 21788 raft_consensus.cc:482] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:47:36.110010 21788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:36.111328 21788 raft_consensus.cc:504] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } }
I20260809 04:47:36.111686 21788 leader_election.cc:302] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f30f2efa53334d89aceeee88f6956f90; no voters:
I20260809 04:47:36.112118 21788 leader_election.cc:288] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:47:36.112365 21796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:47:36.112654 21796 raft_consensus.cc:686] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 1 LEADER]: Becoming Leader. State: Replica: f30f2efa53334d89aceeee88f6956f90, State: Running, Role: LEADER
I20260809 04:47:36.113021 21796 consensus_queue.cc:237] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } }
I20260809 04:47:36.113596 21788 sys_catalog.cc:564] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:36.114403 21796 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f30f2efa53334d89aceeee88f6956f90. Latest consensus state: current_term: 1 leader_uuid: "f30f2efa53334d89aceeee88f6956f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } } }
I20260809 04:47:36.114691 21796 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:36.116454 21797 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f30f2efa53334d89aceeee88f6956f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } } }
I20260809 04:47:36.116636 21803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:47:36.116645 21797 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:36.120702 21803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:47:36.121222 21788 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 04:47:36.135588 21803 catalog_manager.cc:1347] Generated new cluster ID: 5bf6258a5dc04269ac5cb57f88fe4a43
I20260809 04:47:36.135711 21803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:47:36.156131 21803 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:47:36.157596 21803 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:47:36.176237 21803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90: Generated new TSK 0
I20260809 04:47:36.177285 21803 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:47:36.935176 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:38285/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43957
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:37.100971 21836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:37.101315 21836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:37.101383 21836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:37.107431 21836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:37.107573 21836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:37.107681 21836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:47:37.115687 21836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.171.20:38285/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:37.117149 21836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:37.118593 21836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:37.130858 21844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:37.130949 21847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:37.130874 21843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:37.131513 21836 server_base.cc:1034] running on GCE node
I20260809 04:47:37.132084 21836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:37.132921 21836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:37.134130 21836 hybrid_clock.cc:648] HybridClock initialized: now 1786250857134108 us; error 36 us; skew 500 ppm
Aug 09 04:47:37 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:37.137957 21836 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:47:37.139127 21836 webserver.cc:460] Webserver started at http://127.7.171.1:40595/ using document root <none> and password file <none>
I20260809 04:47:37.139915 21836 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:37.140012 21836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:37.140275 21836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:47:37 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:47:37.156885 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "fd68e45302d3406eb3681420c72792c3"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "651f58a4d2923f504f6348ca37219828"
server_key_iv: "a1f73d7505151dbae784016ae464b820"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:37.157567 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "fd68e45302d3406eb3681420c72792c3"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "651f58a4d2923f504f6348ca37219828"
server_key_iv: "a1f73d7505151dbae784016ae464b820"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:37.162731 21836 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:47:37.179404 21854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:37.180877 21836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:47:37.181025 21836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "fd68e45302d3406eb3681420c72792c3"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "651f58a4d2923f504f6348ca37219828"
server_key_iv: "a1f73d7505151dbae784016ae464b820"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:37.181180 21836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:37.211555 21836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:37.215500 21836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:37.215703 21836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:37.216475 21836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:37.217684 21836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:37.217756 21836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:37.217833 21836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:37.217861 21836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:37.229789 21836 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:34105
I20260809 04:47:37.230942 21836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:47:37.232771 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 21836
I20260809 04:47:37.232913 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:47:37.233203 7852 external_mini_cluster.cc:1428] Setting key 4f35728ef8b8157a654962e01d0bb202
I20260809 04:47:37.240561 21982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:34105 every 8 connection(s)
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
Aug 09 04:47:37 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:37.278584 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:37.250016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:53509 (local address 127.7.171.62:43957)
0809 04:47:37.261697 (+ 11681us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:37.261702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:37.261720 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:47:37.261768 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:37.261771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:37.261825 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:37.261926 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:37.266767 (+ 4841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:37.267715 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:37.269703 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:37.270114 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:37.270197 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:37.273644 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:37.273648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:37.273651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:37.273684 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:37.275729 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:37.276493 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:37.276497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:37.276499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:37.276617 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:37.277070 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:37.277074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:37.277075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:37.277326 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:37.277423 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:37.278209 (+ 786us) server_negotiation.cc:299] Negotiation successful
0809 04:47:37.278361 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11586,"thread_start_us":132,"threads_started":1}
I20260809 04:47:37.284315 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:37.250605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43957 (local address 127.7.171.1:53509)
0809 04:47:37.259841 (+ 9236us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:37.259925 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:37.261179 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:37.262456 (+ 1277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:37.262466 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:37.262853 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:37.263791 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:37.263813 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:37.267919 (+ 4106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:37.267923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:37.269517 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:37.269543 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:37.270451 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:37.270454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:37.270686 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:37.271467 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:37.271490 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:37.273392 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:37.275887 (+ 2495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:37.275894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:37.275914 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:37.276360 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:37.276747 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:37.276750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:37.276752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:37.276947 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:37.277497 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:37.277504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:37.277889 (+ 385us) client_negotiation.cc:769] Sending connection context
0809 04:47:37.279179 (+ 1290us) client_negotiation.cc:240] Negotiation successful
0809 04:47:37.279575 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8920,"thread_start_us":151,"threads_started":1}
I20260809 04:47:37.286528 21983 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43957
I20260809 04:47:37.286809 21983 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:37.287459 21983 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
I20260809 04:47:37.289505 21750 ts_manager.cc:194] Registered new tserver with Master: fd68e45302d3406eb3681420c72792c3 (127.7.171.1:34105)
I20260809 04:47:37.291095 21750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:53509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:47:37.311575 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:38285/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43957
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:47:37 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 04:47:37.476310 21990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:37.476794 21990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:37.476917 21990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:37.483203 21990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:37.483368 21990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:37.483516 21990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:47:37.491807 21990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.171.20:38285/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:37.493631 21990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:37.495127 21990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:37.510759 21997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:37.510785 21998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:37.511191 22001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:37.511667 21990 server_base.cc:1034] running on GCE node
I20260809 04:47:37.512192 21990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:37.512988 21990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:37.520181 21990 hybrid_clock.cc:648] HybridClock initialized: now 1786250857520115 us; error 67 us; skew 500 ppm
Aug 09 04:47:37 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:37.524652 21990 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:47:37.525895 21990 webserver.cc:460] Webserver started at http://127.7.171.2:37413/ using document root <none> and password file <none>
I20260809 04:47:37.526692 21990 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:37.526842 21990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:37.527138 21990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:47:37 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:47:37.551754 21990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "1a0de947867e4b868fd16a1111d2cc72"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "d14ac808813921c0961bf4b06e5ff02e"
server_key_iv: "a515cb365c305bd24e552c7ac944187e"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:37.552937 21990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "1a0de947867e4b868fd16a1111d2cc72"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "d14ac808813921c0961bf4b06e5ff02e"
server_key_iv: "a515cb365c305bd24e552c7ac944187e"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:37.558452 21990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:47:37.580664 22008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:37.585456 21990 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.001s
I20260809 04:47:37.585661 21990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "1a0de947867e4b868fd16a1111d2cc72"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "d14ac808813921c0961bf4b06e5ff02e"
server_key_iv: "a515cb365c305bd24e552c7ac944187e"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:37.585831 21990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:37.652446 21990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:37.656203 21990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:37.656396 21990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:37.657095 21990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:37.658130 21990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:37.658192 21990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:37.658243 21990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:37.658272 21990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:37.677436 21990 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:42221
I20260809 04:47:37.678623 21990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:47:37.680552 22131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:42221 every 8 connection(s)
I20260809 04:47:37.681383 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 21990
I20260809 04:47:37.681529 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:47:37.681864 7852 external_mini_cluster.cc:1428] Setting key fb60e222ab130beabc31de9a4475da04
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
Aug 09 04:47:37 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:37.715688 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:37.684744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:43435 (local address 127.7.171.62:43957)
0809 04:47:37.685407 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:37.685413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:37.689305 (+ 3892us) server_negotiation.cc:407] Connection header received
0809 04:47:37.693564 (+ 4259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:37.693568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:37.693636 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:37.693693 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:37.696278 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:37.697126 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:37.704290 (+ 7164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:37.704690 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:37.704783 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:37.711234 (+ 6451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:37.711239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:37.711242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:37.711277 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:37.713059 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:37.713793 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:37.713797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:37.713799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:37.713885 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:37.714293 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:37.714297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:37.714298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:37.714545 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:37.714628 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:37.715384 (+ 756us) server_negotiation.cc:299] Negotiation successful
0809 04:47:37.715527 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
I20260809 04:47:37.720669 22134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:37.683075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43957 (local address 127.7.171.2:43435)
0809 04:47:37.689130 (+ 6055us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:37.689197 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:37.693286 (+ 4089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:37.694152 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:37.694162 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:37.694540 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:37.695226 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:37.695245 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:37.697294 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:37.697297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:37.698725 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:37.698736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:37.705043 (+ 6307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:37.705047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:37.705328 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:37.706129 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:37.706153 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:37.710965 (+ 4812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:37.713230 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:37.713236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:37.713255 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:37.713670 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:37.713988 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:37.713991 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:37.713993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:37.714186 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:37.714721 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:37.714728 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:37.715069 (+ 341us) client_negotiation.cc:769] Sending connection context
0809 04:47:37.716259 (+ 1190us) client_negotiation.cc:240] Negotiation successful
0809 04:47:37.716647 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5789,"thread_start_us":147,"threads_started":1}
I20260809 04:47:37.722461 22132 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43957
I20260809 04:47:37.722728 22132 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:37.723364 22132 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
I20260809 04:47:37.724658 21750 ts_manager.cc:194] Registered new tserver with Master: 1a0de947867e4b868fd16a1111d2cc72 (127.7.171.2:42221)
I20260809 04:47:37.725417 21750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:43435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:47:37.752971 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:38285/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43957
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:47:37 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
W20260809 04:47:37.933511 22138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:37.933883 22138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:37.933952 22138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:37.939880 22138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:37.939957 22138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:37.940071 22138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:47:37.945578 22138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.171.20:38285/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:37.946624 22138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:37.947654 22138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:37.957433 22147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:37.957310 22144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:37.957310 22145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:37.958107 22138 server_base.cc:1034] running on GCE node
I20260809 04:47:37.958499 22138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:37.959148 22138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:37.960292 22138 hybrid_clock.cc:648] HybridClock initialized: now 1786250857960255 us; error 44 us; skew 500 ppm
Aug 09 04:47:37 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:37.963549 22138 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:47:37.964473 22138 webserver.cc:460] Webserver started at http://127.7.171.3:34849/ using document root <none> and password file <none>
I20260809 04:47:37.965031 22138 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:37.965124 22138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:37.965334 22138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:47:37 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:47:37.980136 22138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "fcc6747e1504414dbb91c6153eeb644c"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "639a6b3a0d81fe82089cae6f2a1131f4"
server_key_iv: "c5ffe2fbedeee2ea1dd8a1bc36d6a8ea"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:37.980976 22138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "fcc6747e1504414dbb91c6153eeb644c"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "639a6b3a0d81fe82089cae6f2a1131f4"
server_key_iv: "c5ffe2fbedeee2ea1dd8a1bc36d6a8ea"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:37.986340 22138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 04:47:38.004891 22154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:38.007556 22138 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260809 04:47:38.007825 22138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "fcc6747e1504414dbb91c6153eeb644c"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "639a6b3a0d81fe82089cae6f2a1131f4"
server_key_iv: "c5ffe2fbedeee2ea1dd8a1bc36d6a8ea"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:38.007997 22138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:38.029083 22138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:38.032509 22138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:38.032729 22138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:38.033427 22138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:38.034371 22138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:38.034448 22138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:38.034514 22138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:38.034575 22138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:38.046550 22138 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:39049
I20260809 04:47:38.047617 22138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:47:38.048837 22278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:39049 every 8 connection(s)
I20260809 04:47:38.050591 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 22138
I20260809 04:47:38.050732 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:47:38.051065 7852 external_mini_cluster.cc:1428] Setting key 49b0411027abd4a822b68445003b1bde
Aug 09 04:47:38 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:38.071388 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:38.049990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:34225 (local address 127.7.171.62:43957)
0809 04:47:38.050129 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:38.050134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:38.056646 (+ 6512us) server_negotiation.cc:407] Connection header received
0809 04:47:38.057922 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:38.057926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:38.057983 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:38.058037 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:38.059887 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.060729 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.062755 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.063118 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.063208 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:38.066601 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:38.066606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:38.066609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:38.066637 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:38.068628 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:38.069332 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:38.069336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:38.069338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:38.069437 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:38.069909 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:38.069913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:38.069914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:38.070165 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:38.070238 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:38.071062 (+ 824us) server_negotiation.cc:299] Negotiation successful
0809 04:47:38.071210 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:47:38.073303 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:38.050535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43957 (local address 127.7.171.3:34225)
0809 04:47:38.056449 (+ 5914us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:38.056519 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:38.057652 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:38.058518 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:38.058528 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:38.058941 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:38.059681 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:38.059712 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.060912 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.060916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:38.062594 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:38.062607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.063448 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.063453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:38.063712 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:38.064551 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:38.064574 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:38.066394 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:38.068766 (+ 2372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:38.068773 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:38.068789 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:38.069196 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:38.069571 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:38.069574 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:38.069576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:38.069780 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:38.070342 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:38.070349 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:38.070723 (+ 374us) client_negotiation.cc:769] Sending connection context
0809 04:47:38.071666 (+ 943us) client_negotiation.cc:240] Negotiation successful
0809 04:47:38.072115 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5654,"thread_start_us":147,"threads_started":1}
I20260809 04:47:38.074681 22279 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43957
I20260809 04:47:38.075052 22279 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:38.075706 22279 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
I20260809 04:47:38.077018 21750 ts_manager.cc:194] Registered new tserver with Master: fcc6747e1504414dbb91c6153eeb644c (127.7.171.3:39049)
I20260809 04:47:38.077781 21750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:34225
I20260809 04:47:38.079319 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:47:38.080142 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 21836
I20260809 04:47:38.094087 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 21990
I20260809 04:47:38.104561 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 22138
I20260809 04:47:38.112421 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 21715
I20260809 04:47:38.122021 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43957
--webserver_interface=127.7.171.62
--webserver_port=42757
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:43957
--ranger_config_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:38285/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.171.20:38285/kms with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:38.236691 22282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:38.236963 22282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:38.237005 22282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:38.241127 22282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:38.241194 22282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:38.241212 22282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:38.241230 22282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:38.241245 22282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:38.247156 22282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.171.20:38285/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:43957
--ranger_config_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=42757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:38.248507 22282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:38.249965 22282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:38.259863 22289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:38.259924 22288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:38.259898 22291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:38.260113 22282 server_base.cc:1034] running on GCE node
I20260809 04:47:38.260649 22282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:38.261672 22282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:38.262828 22282 hybrid_clock.cc:648] HybridClock initialized: now 1786250858262795 us; error 47 us; skew 500 ppm
Aug 09 04:47:38 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:38.265620 22282 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:47:38.266587 22282 webserver.cc:460] Webserver started at http://127.7.171.62:42757/ using document root <none> and password file <none>
I20260809 04:47:38.267166 22282 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:38.267272 22282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:47:38 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:47:38.313313 22282 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:47:38.315696 22299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:38.317101 22282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:47:38.317265 22282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "f30f2efa53334d89aceeee88f6956f90"
format_stamp: "Formatted at 2026-08-09 04:47:35 on dist-test-slave-1zqn"
server_key: "935c48915646f468b3fd5cb2a7666cbb"
server_key_iv: "f1d74c296d880bdc04f935360735565c"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:38.317830 22282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:38.342854 22282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:38.345716 22282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:38.345944 22282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:38.353756 22355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:43957 every 8 connection(s)
I20260809 04:47:38.353767 22282 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:43957
I20260809 04:47:38.354969 22282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:47:38.359390 22356 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:47:38.360642 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 22282
I20260809 04:47:38.361111 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:38285/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:34105
--local_ip_for_outbound_sockets=127.7.171.1
--tserver_master_addrs=127.7.171.62:43957
--webserver_port=40595
--webserver_interface=127.7.171.1
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.171.20:38285/kms with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:47:38.361217 22356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90: Bootstrap starting.
I20260809 04:47:38.379498 22356 log.cc:824] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:38.386334 22356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:47:38.386765 22356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90: Bootstrap complete.
I20260809 04:47:38.391072 22356 raft_consensus.cc:348] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } }
I20260809 04:47:38.391485 22356 raft_consensus.cc:729] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30f2efa53334d89aceeee88f6956f90, State: Initialized, Role: FOLLOWER
I20260809 04:47:38.391976 22356 consensus_queue.cc:260] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } }
I20260809 04:47:38.392149 22356 raft_consensus.cc:388] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:47:38.392246 22356 raft_consensus.cc:482] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:47:38.392371 22356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:47:38.394320 22356 raft_consensus.cc:504] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } }
I20260809 04:47:38.394671 22356 leader_election.cc:302] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f30f2efa53334d89aceeee88f6956f90; no voters:
I20260809 04:47:38.395036 22356 leader_election.cc:288] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:47:38.395133 22361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:47:38.395318 22361 raft_consensus.cc:686] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [term 2 LEADER]: Becoming Leader. State: Replica: f30f2efa53334d89aceeee88f6956f90, State: Running, Role: LEADER
I20260809 04:47:38.395607 22361 consensus_queue.cc:237] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } }
I20260809 04:47:38.396322 22356 sys_catalog.cc:564] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:38.397003 22362 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f30f2efa53334d89aceeee88f6956f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } } }
I20260809 04:47:38.397140 22362 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:38.397738 22363 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f30f2efa53334d89aceeee88f6956f90. Latest consensus state: current_term: 2 leader_uuid: "f30f2efa53334d89aceeee88f6956f90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2efa53334d89aceeee88f6956f90" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43957 } } }
I20260809 04:47:38.397886 22363 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:38.398152 22369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:47:38.398697 22356 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 04:47:38.404740 22369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:47:38.405592 22369 catalog_manager.cc:1259] Loaded cluster ID: 5bf6258a5dc04269ac5cb57f88fe4a43
I20260809 04:47:38.405671 22369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:47:38.408921 22369 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:47:38.409646 22369 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90: Loaded TSK: 0
I20260809 04:47:38.412132 22369 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:47:38.520438 22359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:38.520800 22359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:38.520879 22359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:38.527347 22359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:38.527428 22359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:38.527524 22359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:47:38.535517 22359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.171.20:38285/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:34105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=40595
--tserver_master_addrs=127.7.171.62:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:38.536979 22359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:38.538417 22359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:38.552249 22398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:38.552603 22400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:38.553596 22397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:38.553781 22359 server_base.cc:1034] running on GCE node
I20260809 04:47:38.554270 22359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:38.555130 22359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:38.556315 22359 hybrid_clock.cc:648] HybridClock initialized: now 1786250858556287 us; error 48 us; skew 500 ppm
Aug 09 04:47:38 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:38.560110 22359 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:47:38.561174 22359 webserver.cc:460] Webserver started at http://127.7.171.1:40595/ using document root <none> and password file <none>
I20260809 04:47:38.561863 22359 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:38.561986 22359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:47:38 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:47:38.586645 22359 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:47:38.589784 22409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:38.591185 22359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:47:38.591334 22359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "fd68e45302d3406eb3681420c72792c3"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "651f58a4d2923f504f6348ca37219828"
server_key_iv: "a1f73d7505151dbae784016ae464b820"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:38.591936 22359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:38.616796 22359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:38.620877 22359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:38.621104 22359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:38.621958 22359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:38.623113 22359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:38.623181 22359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:38.623245 22359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:38.623276 22359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:38.635224 22359 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:34105
I20260809 04:47:38.635383 22533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:34105 every 8 connection(s)
I20260809 04:47:38.636456 22359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:47:38.638902 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 22359
I20260809 04:47:38.639473 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:38285/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:42221
--local_ip_for_outbound_sockets=127.7.171.2
--tserver_master_addrs=127.7.171.62:43957
--webserver_port=37413
--webserver_interface=127.7.171.2
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.171.20:38285/kms with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:47:38 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:38.658203 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:38.638917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43957 (local address 127.7.171.1:59211)
0809 04:47:38.639979 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:38.640022 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:38.641059 (+ 1037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:38.643076 (+ 2017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:38.643168 (+ 92us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:38.643533 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:38.644229 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:38.644250 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.646171 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.646174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:38.647594 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:38.647607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.648944 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.648949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:38.649160 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:38.649904 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:38.649926 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:38.651602 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:38.654220 (+ 2618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:38.654226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:38.654242 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:38.654608 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:38.655034 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:38.655037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:38.655039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:38.655228 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:38.656023 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:38.656032 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:38.656644 (+ 612us) client_negotiation.cc:769] Sending connection context
0809 04:47:38.656855 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 04:47:38.657184 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":139,"threads_started":1}
I20260809 04:47:38.660025 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:38.638758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:59211 (local address 127.7.171.62:43957)
0809 04:47:38.639403 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:38.639428 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:38.641362 (+ 1934us) server_negotiation.cc:407] Connection header received
0809 04:47:38.641725 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:38.641743 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:38.642173 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:38.642541 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:38.644438 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.646002 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.647746 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.648505 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.648702 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:38.651826 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:38.651834 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:38.651851 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:38.651884 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:38.654067 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:38.654756 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:38.654763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:38.654769 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:38.654914 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:38.655432 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:38.655435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:38.655436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:38.655804 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:38.655953 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:38.658810 (+ 2857us) server_negotiation.cc:299] Negotiation successful
0809 04:47:38.659105 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":120,"threads_started":1}
I20260809 04:47:38.663618 22534 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43957
I20260809 04:47:38.663843 22534 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:38.664389 22534 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
W20260809 04:47:38.666854 22534 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:43957 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:47:38.667438 22534 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:38.667652 22534 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
I20260809 04:47:38.668639 22534 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:38.668800 22534 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
W20260809 04:47:38.669605 22534 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:47:38.782366 22541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:38.782661 22541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:38.782753 22541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:38.787377 22541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:38.787468 22541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:38.787576 22541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:47:38.793632 22541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.171.20:38285/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:42221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=37413
--tserver_master_addrs=127.7.171.62:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:38.794839 22541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:38.796027 22541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:38.806641 22547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:38.806605 22550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:38.806602 22548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:38.807211 22541 server_base.cc:1034] running on GCE node
I20260809 04:47:38.807886 22541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:38.808832 22541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:38.810070 22541 hybrid_clock.cc:648] HybridClock initialized: now 1786250858810026 us; error 50 us; skew 500 ppm
Aug 09 04:47:38 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:38.814111 22541 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:47:38.815611 22541 webserver.cc:460] Webserver started at http://127.7.171.2:37413/ using document root <none> and password file <none>
I20260809 04:47:38.816540 22541 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:38.816672 22541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:47:38 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:47:38.845603 22541 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.003s sys 0.000s
I20260809 04:47:38.854568 22560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:38.856981 22541 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.004s sys 0.000s
I20260809 04:47:38.857149 22541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "1a0de947867e4b868fd16a1111d2cc72"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "d14ac808813921c0961bf4b06e5ff02e"
server_key_iv: "a515cb365c305bd24e552c7ac944187e"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:38.857939 22541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:38.880827 22541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:38.884895 22541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:38.885138 22541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:38.885898 22541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:38.887005 22541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:38.887073 22541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:38.887143 22541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:38.887176 22541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:38.899861 22541 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:42221
I20260809 04:47:38.899881 22691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:42221 every 8 connection(s)
I20260809 04:47:38.901245 22541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:47:38.909512 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 22541
I20260809 04:47:38.910086 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.171.20:38285/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:39049
--local_ip_for_outbound_sockets=127.7.171.3
--tserver_master_addrs=127.7.171.62:43957
--webserver_port=34849
--webserver_interface=127.7.171.3
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.171.20:38285/kms with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:47:38 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:38.920940 22695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:38.903776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43957 (local address 127.7.171.2:58853)
0809 04:47:38.904303 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:38.904357 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:38.905251 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:38.905790 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:38.905797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:38.906071 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:38.906541 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:38.906555 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.907525 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.907529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:38.908472 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:38.908482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.909504 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.909509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:38.909770 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:38.910538 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:38.910563 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:38.914336 (+ 3773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:38.916561 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:38.916570 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:38.916593 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:38.916986 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:38.917327 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:38.917333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:38.917338 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:38.917535 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:38.918092 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:38.918101 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:38.918508 (+ 407us) client_negotiation.cc:769] Sending connection context
0809 04:47:38.918771 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 04:47:38.919181 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":151,"threads_started":1}
I20260809 04:47:38.921833 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:38.903332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:58853 (local address 127.7.171.62:43957)
0809 04:47:38.904803 (+ 1471us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:38.904810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:38.904828 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:47:38.905469 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:38.905472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:38.905528 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:38.905608 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:38.906706 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.907387 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.908601 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:38.908846 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:38.908959 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:38.914545 (+ 5586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:38.914550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:38.914554 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:38.914593 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:38.916428 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:38.917114 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:38.917120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:38.917125 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:38.917221 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:38.917664 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:38.917670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:38.917674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:38.917937 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:38.918034 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:38.921370 (+ 3336us) server_negotiation.cc:299] Negotiation successful
0809 04:47:38.921592 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1374}
I20260809 04:47:38.923756 22692 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43957
I20260809 04:47:38.924028 22692 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:38.924746 22692 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
W20260809 04:47:38.926787 22692 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:43957 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:47:38.926923 22692 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:38.927127 22692 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
I20260809 04:47:38.928207 22692 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:38.928388 22692 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
W20260809 04:47:38.929343 22692 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:47:39.061782 22696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:39.062148 22696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:39.062234 22696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:39.066783 22696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:39.066882 22696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:39.066979 22696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:47:39.073007 22696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.171.20:38285/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:39049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=34849
--tserver_master_addrs=127.7.171.62:43957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:39.074191 22696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:39.075276 22696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:39.088392 22706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:39.088513 22703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:39.088933 22704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:39.089047 22696 server_base.cc:1034] running on GCE node
I20260809 04:47:39.089732 22696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:39.090595 22696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:39.093904 22696 hybrid_clock.cc:648] HybridClock initialized: now 1786250859093821 us; error 97 us; skew 500 ppm
Aug 09 04:47:39 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250859, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:39.097960 22696 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:47:39.099159 22696 webserver.cc:460] Webserver started at http://127.7.171.3:34849/ using document root <none> and password file <none>
I20260809 04:47:39.099896 22696 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:39.100003 22696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:47:39 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250859, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:47:39.126121 22696 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:47:39.129148 22716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:39.130617 22696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:47:39.130739 22696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "fcc6747e1504414dbb91c6153eeb644c"
format_stamp: "Formatted at 2026-08-09 04:47:37 on dist-test-slave-1zqn"
server_key: "639a6b3a0d81fe82089cae6f2a1131f4"
server_key_iv: "c5ffe2fbedeee2ea1dd8a1bc36d6a8ea"
server_key_version: "kuduclusterkey@0"
I20260809 04:47:39.131364 22696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:39.147264 22696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:39.150710 22696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:39.150930 22696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:39.151634 22696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:39.152688 22696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:39.152735 22696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:39.152776 22696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:39.152827 22696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:39.166160 22696 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:39049
I20260809 04:47:39.166201 22845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:39049 every 8 connection(s)
I20260809 04:47:39.167608 22696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:47:39.173172 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 22696
Aug 09 04:47:39 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250859, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:39.183457 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:39.170085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:46433 (local address 127.7.171.62:43957)
0809 04:47:39.170234 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:39.170239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:39.171414 (+ 1175us) server_negotiation.cc:407] Connection header received
0809 04:47:39.172563 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:39.172566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:39.172628 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:39.172744 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:39.174215 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:39.174956 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:39.176186 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:39.176514 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:39.176609 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:39.179361 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:39.179365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:39.179368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:39.179392 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:39.181219 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:39.181848 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:39.181851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:39.181852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:39.181950 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:39.182300 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:39.182305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:39.182306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:39.182485 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:39.182595 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:39.183103 (+ 508us) server_negotiation.cc:299] Negotiation successful
0809 04:47:39.183231 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:47:39.184401 22849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:39.170609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43957 (local address 127.7.171.3:46433)
0809 04:47:39.171236 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:39.171298 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:39.172325 (+ 1027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:39.172926 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:39.172937 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:39.173319 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:39.174035 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:39.174056 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:39.175091 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:39.175094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:39.175979 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:39.175987 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:39.176648 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:39.176652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:39.176865 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:39.177612 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:39.177638 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:39.179191 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:39.181340 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:39.181345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:39.181362 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:39.181720 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:39.182046 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:39.182050 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:39.182052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:39.182197 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:39.182594 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:39.182601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:39.182861 (+ 260us) client_negotiation.cc:769] Sending connection context
0809 04:47:39.183063 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 04:47:39.183395 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":152,"threads_started":1}
I20260809 04:47:39.185729 22846 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43957
I20260809 04:47:39.186008 22846 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:39.186641 22846 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
W20260809 04:47:39.188405 22846 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:43957 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:47:39.188525 22846 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:39.188685 22846 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
I20260809 04:47:39.188772 22851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:39.174282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.7.171.62:43957)
0809 04:47:39.176301 (+ 2019us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:39.176305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:39.176316 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:47:39.176346 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:39.176348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:39.176389 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:39.176470 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:39.178237 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:39.179532 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:39.180908 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:39.181318 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:39.181453 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:39.183246 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:39.183250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:39.183253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:39.183282 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:39.185766 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:39.186322 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:39.186325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:39.186327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:39.186423 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:39.186808 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:39.186811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:39.186813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:39.187135 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:39.188216 (+ 1081us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:39.188299 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 04:47:39.188432 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1949,"thread_start_us":87,"threads_started":1}
I20260809 04:47:39.189695 22846 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:39.189913 22846 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
W20260809 04:47:39.190688 22846 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 04:47:39 dist-test-slave-1zqn krb5kdc[21239](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250859, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:39.669852 22534 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:39.670195 22534 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
I20260809 04:47:39.671579 22316 ts_manager.cc:194] Registered new tserver with Master: fd68e45302d3406eb3681420c72792c3 (127.7.171.1:34105)
I20260809 04:47:39.673345 22316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:59211
I20260809 04:47:39.929574 22692 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:39.929903 22692 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
I20260809 04:47:39.930691 22316 ts_manager.cc:194] Registered new tserver with Master: 1a0de947867e4b868fd16a1111d2cc72 (127.7.171.2:42221)
I20260809 04:47:39.931414 22316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:58853
Aug 09 04:47:39 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250859, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:47:40.190924 22846 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:40.191274 22846 heartbeater.cc:511] Master 127.7.171.62:43957 requested a full tablet report, sending...
I20260809 04:47:40.192042 22316 ts_manager.cc:194] Registered new tserver with Master: fcc6747e1504414dbb91c6153eeb644c (127.7.171.3:39049)
I20260809 04:47:40.192900 22316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:46433
I20260809 04:47:40.203256 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:47:40.217017 22869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.205757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52268 (local address 127.7.171.62:43957)
0809 04:47:40.206219 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:40.206225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:40.206465 (+ 240us) server_negotiation.cc:407] Connection header received
0809 04:47:40.206698 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:40.206703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:40.206758 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:40.206821 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:40.207899 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.208784 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.209656 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.209908 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.210027 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.212608 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:40.212613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:40.212615 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:40.212646 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:40.214636 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:40.215180 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:40.215184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:40.215186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:40.215280 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:40.215582 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:40.215587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:40.215589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:40.215824 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:40.215968 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:40.216735 (+ 767us) server_negotiation.cc:299] Negotiation successful
0809 04:47:40.216879 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":82,"threads_started":1}
I20260809 04:47:40.221294 22316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:47:40.225837 22316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 09 04:47:40 dist-test-slave-1zqn krb5kdc[21239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250859, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for HTTP/127.7.171.20@KRBTEST.COM
I20260809 04:47:40.253275 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.241214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37478 (local address 127.7.171.3:39049)
0809 04:47:40.244345 (+ 3131us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:40.244356 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:40.245986 (+ 1630us) server_negotiation.cc:407] Connection header received
0809 04:47:40.246432 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:40.246439 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:40.246654 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:40.247015 (+ 361us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:40.248083 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.249541 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.251334 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.252218 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.252356 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.252521 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:40.252992 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 04:47:40.253108 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3000,"thread_start_us":126,"threads_started":1}
I20260809 04:47:40.254266 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.240989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:39049 (local address 127.0.0.1:37478)
0809 04:47:40.245829 (+ 4840us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:40.245844 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:40.246254 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:40.247336 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:40.247338 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:40.247357 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:40.247808 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:40.247818 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.249713 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.249716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:40.251183 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:40.251195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.252645 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.252648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:40.252762 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.252804 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 04:47:40.253960 (+ 1156us) client_negotiation.cc:240] Negotiation successful
0809 04:47:40.254111 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4745,"mutex_wait_us":799,"thread_start_us":107,"threads_started":1}
I20260809 04:47:40.257408 22750 tablet_service.cc:1467] Processing CreateTablet for tablet 0cac2be219b64e88bcac09e61539f0f7 (DEFAULT_TABLE table=test-table [id=e34143595e2c477d970bff44e150b074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:40.258574 22750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cac2be219b64e88bcac09e61539f0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:40.269079 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.241171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.7.171.2:42221)
0809 04:47:40.244559 (+ 3388us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:40.244567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:40.244598 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:47:40.244693 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:40.244701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:40.246331 (+ 1630us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:40.246513 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:40.247926 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.250601 (+ 2675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.252246 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.263834 (+ 11588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.263961 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.264167 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:40.268743 (+ 4576us) server_negotiation.cc:299] Negotiation successful
0809 04:47:40.268894 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3277,"thread_start_us":117,"threads_started":1}
I20260809 04:47:40.269548 22881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.245189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:34105 (local address 127.0.0.1:38624)
0809 04:47:40.254418 (+ 9229us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:40.254432 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:40.254520 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:40.265090 (+ 10570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:40.265093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:40.265115 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:40.265240 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:40.265247 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.266813 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.266817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:40.267981 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:40.267993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.269138 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.269142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:40.269231 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.269257 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:47:40.269313 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 04:47:40.269405 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9154,"thread_start_us":133,"threads_started":1}
I20260809 04:47:40.270321 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.245348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.7.171.1:34105)
0809 04:47:40.264427 (+ 19079us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:40.264435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:40.264463 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:47:40.264551 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:40.264557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:40.264907 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:40.265405 (+ 498us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:40.265567 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.266671 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.268194 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.268952 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.269832 (+ 880us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.270019 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:40.270074 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 04:47:40.270183 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18939,"thread_start_us":76,"threads_started":1}
I20260809 04:47:40.273885 22455 tablet_service.cc:1467] Processing CreateTablet for tablet 0cac2be219b64e88bcac09e61539f0f7 (DEFAULT_TABLE table=test-table [id=e34143595e2c477d970bff44e150b074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:40.275044 22455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cac2be219b64e88bcac09e61539f0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:40.277935 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.240988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:42221 (local address 127.0.0.1:32820)
0809 04:47:40.241959 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:40.242020 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:40.242179 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:40.246749 (+ 4570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:40.246759 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:40.246792 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:40.247741 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:40.247751 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.250764 (+ 3013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.250768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:40.252082 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:40.252094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.268382 (+ 16288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.268387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:40.268511 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.268547 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 04:47:40.277588 (+ 9041us) client_negotiation.cc:240] Negotiation successful
0809 04:47:40.277728 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":2,"thread_start_us":4569,"threads_started":1}
I20260809 04:47:40.279699 22882 tablet_bootstrap.cc:492] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: Bootstrap starting.
I20260809 04:47:40.282353 22610 tablet_service.cc:1467] Processing CreateTablet for tablet 0cac2be219b64e88bcac09e61539f0f7 (DEFAULT_TABLE table=test-table [id=e34143595e2c477d970bff44e150b074]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:40.283038 22882 tablet_bootstrap.cc:654] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:40.283582 22610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cac2be219b64e88bcac09e61539f0f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:40.283978 22882 log.cc:824] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:40.283849 22883 tablet_bootstrap.cc:492] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3: Bootstrap starting.
I20260809 04:47:40.289114 22883 tablet_bootstrap.cc:654] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:40.290892 22883 log.cc:824] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:40.296779 22882 tablet_bootstrap.cc:492] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: No bootstrap required, opened a new log
I20260809 04:47:40.297060 22882 ts_tablet_manager.cc:1397] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.000s
I20260809 04:47:40.297093 22883 tablet_bootstrap.cc:492] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3: No bootstrap required, opened a new log
I20260809 04:47:40.297341 22883 ts_tablet_manager.cc:1397] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260809 04:47:40.301952 22883 raft_consensus.cc:348] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd68e45302d3406eb3681420c72792c3" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34105 } } peers { permanent_uuid: "fcc6747e1504414dbb91c6153eeb644c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 39049 } } peers { permanent_uuid: "1a0de947867e4b868fd16a1111d2cc72" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42221 } }
I20260809 04:47:40.302325 22883 raft_consensus.cc:374] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:40.302448 22883 raft_consensus.cc:729] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd68e45302d3406eb3681420c72792c3, State: Initialized, Role: FOLLOWER
I20260809 04:47:40.303016 22883 consensus_queue.cc:260] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd68e45302d3406eb3681420c72792c3" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34105 } } peers { permanent_uuid: "fcc6747e1504414dbb91c6153eeb644c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 39049 } } peers { permanent_uuid: "1a0de947867e4b868fd16a1111d2cc72" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42221 } }
I20260809 04:47:40.303318 22882 raft_consensus.cc:348] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd68e45302d3406eb3681420c72792c3" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34105 } } peers { permanent_uuid: "fcc6747e1504414dbb91c6153eeb644c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 39049 } } peers { permanent_uuid: "1a0de947867e4b868fd16a1111d2cc72" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42221 } }
I20260809 04:47:40.304371 22883 ts_tablet_manager.cc:1428] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260809 04:47:40.304863 22534 heartbeater.cc:503] Master 127.7.171.62:43957 was elected leader, sending a full tablet report...
I20260809 04:47:40.304194 22882 raft_consensus.cc:374] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:40.305713 22882 raft_consensus.cc:729] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc6747e1504414dbb91c6153eeb644c, State: Initialized, Role: FOLLOWER
I20260809 04:47:40.306533 22882 consensus_queue.cc:260] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd68e45302d3406eb3681420c72792c3" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34105 } } peers { permanent_uuid: "fcc6747e1504414dbb91c6153eeb644c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 39049 } } peers { permanent_uuid: "1a0de947867e4b868fd16a1111d2cc72" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42221 } }
I20260809 04:47:40.308353 22882 ts_tablet_manager.cc:1428] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260809 04:47:40.308774 22846 heartbeater.cc:503] Master 127.7.171.62:43957 was elected leader, sending a full tablet report...
I20260809 04:47:40.314231 22886 tablet_bootstrap.cc:492] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: Bootstrap starting.
I20260809 04:47:40.316725 22886 tablet_bootstrap.cc:654] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:40.317723 22886 log.cc:824] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:40.323668 22886 tablet_bootstrap.cc:492] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: No bootstrap required, opened a new log
I20260809 04:47:40.323935 22886 ts_tablet_manager.cc:1397] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.000s
I20260809 04:47:40.328696 22886 raft_consensus.cc:348] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd68e45302d3406eb3681420c72792c3" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34105 } } peers { permanent_uuid: "fcc6747e1504414dbb91c6153eeb644c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 39049 } } peers { permanent_uuid: "1a0de947867e4b868fd16a1111d2cc72" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42221 } }
I20260809 04:47:40.329069 22886 raft_consensus.cc:374] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:40.329164 22886 raft_consensus.cc:729] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a0de947867e4b868fd16a1111d2cc72, State: Initialized, Role: FOLLOWER
I20260809 04:47:40.329703 22886 consensus_queue.cc:260] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd68e45302d3406eb3681420c72792c3" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34105 } } peers { permanent_uuid: "fcc6747e1504414dbb91c6153eeb644c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 39049 } } peers { permanent_uuid: "1a0de947867e4b868fd16a1111d2cc72" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42221 } }
I20260809 04:47:40.330886 22886 ts_tablet_manager.cc:1428] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 04:47:40.331338 22692 heartbeater.cc:503] Master 127.7.171.62:43957 was elected leader, sending a full tablet report...
W20260809 04:47:40.388972 22535 tablet.cc:2369] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:47:40.403818 22693 tablet.cc:2369] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:47:40.419873 22847 tablet.cc:2369] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:47:40.496285 22891 raft_consensus.cc:482] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:40.496547 22891 raft_consensus.cc:504] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd68e45302d3406eb3681420c72792c3" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34105 } } peers { permanent_uuid: "fcc6747e1504414dbb91c6153eeb644c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 39049 } } peers { permanent_uuid: "1a0de947867e4b868fd16a1111d2cc72" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42221 } }
I20260809 04:47:40.498097 22891 leader_election.cc:288] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd68e45302d3406eb3681420c72792c3 (127.7.171.1:34105), fcc6747e1504414dbb91c6153eeb644c (127.7.171.3:39049)
I20260809 04:47:40.502959 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.498559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:60301 (local address 127.7.171.3:39049)
0809 04:47:40.498748 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:40.498754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:40.499230 (+ 476us) server_negotiation.cc:407] Connection header received
0809 04:47:40.499303 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:40.499306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:40.499361 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:40.499443 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:40.499864 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.500599 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.501723 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.502301 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.502408 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.502477 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:40.502735 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 04:47:40.502823 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 04:47:40.503122 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.498783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:55321 (local address 127.7.171.1:34105)
0809 04:47:40.498899 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:40.498905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:40.498924 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:47:40.499090 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:40.499093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:40.499166 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:40.499244 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:40.499589 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.500359 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.501953 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.502536 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.502662 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.502719 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:40.502902 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 04:47:40.502973 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 04:47:40.503397 22902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.498366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:39049 (local address 127.7.171.2:60301)
0809 04:47:40.499091 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:40.499105 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:40.499225 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:40.499566 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:40.499569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:40.499601 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:40.499727 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:40.499734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.500713 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.500718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:40.501586 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:40.501595 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.502448 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.502452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:40.502557 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.502596 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 04:47:40.503173 (+ 577us) client_negotiation.cc:240] Negotiation successful
0809 04:47:40.503248 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":22,"thread_start_us":88,"threads_started":1}
I20260809 04:47:40.503429 22901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.498366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:34105 (local address 127.7.171.2:55321)
0809 04:47:40.498798 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:40.498815 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:40.498946 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:40.499327 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:40.499330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:40.499346 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:40.499476 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:40.499480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.500496 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.500500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:40.501820 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:40.501832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.502666 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.502670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:40.502768 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.502817 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 04:47:40.502886 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 04:47:40.502954 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":113,"threads_started":1}
I20260809 04:47:40.504199 22774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cac2be219b64e88bcac09e61539f0f7" candidate_uuid: "1a0de947867e4b868fd16a1111d2cc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcc6747e1504414dbb91c6153eeb644c" is_pre_election: true
I20260809 04:47:40.504248 22479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cac2be219b64e88bcac09e61539f0f7" candidate_uuid: "1a0de947867e4b868fd16a1111d2cc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd68e45302d3406eb3681420c72792c3" is_pre_election: true
I20260809 04:47:40.504561 22774 raft_consensus.cc:2454] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a0de947867e4b868fd16a1111d2cc72 in term 0.
I20260809 04:47:40.504560 22479 raft_consensus.cc:2454] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a0de947867e4b868fd16a1111d2cc72 in term 0.
I20260809 04:47:40.505119 22564 leader_election.cc:302] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a0de947867e4b868fd16a1111d2cc72, fd68e45302d3406eb3681420c72792c3; no voters:
I20260809 04:47:40.505425 22891 raft_consensus.cc:2781] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:47:40.505534 22891 raft_consensus.cc:482] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:47:40.505604 22891 raft_consensus.cc:3037] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:40.507031 22891 raft_consensus.cc:504] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd68e45302d3406eb3681420c72792c3" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34105 } } peers { permanent_uuid: "fcc6747e1504414dbb91c6153eeb644c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 39049 } } peers { permanent_uuid: "1a0de947867e4b868fd16a1111d2cc72" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42221 } }
I20260809 04:47:40.507439 22891 leader_election.cc:288] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [CANDIDATE]: Term 1 election: Requested vote from peers fd68e45302d3406eb3681420c72792c3 (127.7.171.1:34105), fcc6747e1504414dbb91c6153eeb644c (127.7.171.3:39049)
I20260809 04:47:40.507826 22479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cac2be219b64e88bcac09e61539f0f7" candidate_uuid: "1a0de947867e4b868fd16a1111d2cc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd68e45302d3406eb3681420c72792c3"
I20260809 04:47:40.507964 22479 raft_consensus.cc:3037] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:40.509295 22479 raft_consensus.cc:2454] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a0de947867e4b868fd16a1111d2cc72 in term 1.
I20260809 04:47:40.509560 22774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cac2be219b64e88bcac09e61539f0f7" candidate_uuid: "1a0de947867e4b868fd16a1111d2cc72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcc6747e1504414dbb91c6153eeb644c"
I20260809 04:47:40.509601 22564 leader_election.cc:302] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a0de947867e4b868fd16a1111d2cc72, fd68e45302d3406eb3681420c72792c3; no voters:
I20260809 04:47:40.509696 22774 raft_consensus.cc:3037] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:40.509810 22891 raft_consensus.cc:2781] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:47:40.510082 22891 raft_consensus.cc:686] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 1 LEADER]: Becoming Leader. State: Replica: 1a0de947867e4b868fd16a1111d2cc72, State: Running, Role: LEADER
I20260809 04:47:40.510430 22891 consensus_queue.cc:237] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd68e45302d3406eb3681420c72792c3" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34105 } } peers { permanent_uuid: "fcc6747e1504414dbb91c6153eeb644c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 39049 } } peers { permanent_uuid: "1a0de947867e4b868fd16a1111d2cc72" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42221 } }
I20260809 04:47:40.510947 22774 raft_consensus.cc:2454] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a0de947867e4b868fd16a1111d2cc72 in term 1.
I20260809 04:47:40.514151 22317 catalog_manager.cc:5495] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a0de947867e4b868fd16a1111d2cc72 (127.7.171.2). New cstate: current_term: 1 leader_uuid: "1a0de947867e4b868fd16a1111d2cc72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd68e45302d3406eb3681420c72792c3" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcc6747e1504414dbb91c6153eeb644c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 39049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a0de947867e4b868fd16a1111d2cc72" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42221 } health_report { overall_health: HEALTHY } } }
I20260809 04:47:40.543767 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.540002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58012 (local address 127.7.171.2:42221)
0809 04:47:40.540165 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:40.540171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:40.540187 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:47:40.540303 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:40.540305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:40.540365 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:40.540482 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:47:40.540743 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.541255 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.542185 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:40.542414 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:40.542531 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:40.542863 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:47:40.543303 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:47:40.543450 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:40.543517 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 04:47:40.543579 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:47:40.550225 22479 raft_consensus.cc:1261] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3 [term 1 FOLLOWER]: Refusing update from remote peer 1a0de947867e4b868fd16a1111d2cc72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:40.550268 22774 raft_consensus.cc:1261] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c [term 1 FOLLOWER]: Refusing update from remote peer 1a0de947867e4b868fd16a1111d2cc72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:40.550995 22905 consensus_queue.cc:1035] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd68e45302d3406eb3681420c72792c3" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:40.551180 22891 consensus_queue.cc:1035] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcc6747e1504414dbb91c6153eeb644c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:40.560956 22910 mvcc.cc:205] Tried to move back new op lower bound from 7316483524805877760 to 7316483524654448640. Current Snapshot: MvccSnapshot[applied={T|T < 7316483524805877760}]
I20260809 04:47:40.561110 22909 mvcc.cc:205] Tried to move back new op lower bound from 7316483524805877760 to 7316483524654448640. Current Snapshot: MvccSnapshot[applied={T|T < 7316483524805877760}]
I20260809 04:47:40.572151 22315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:40.572361 22315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:40.575253 22315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f30f2efa53334d89aceeee88f6956f90: Sending DeleteTablet for 3 replicas of tablet 0cac2be219b64e88bcac09e61539f0f7
I20260809 04:47:40.577193 22455 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cac2be219b64e88bcac09e61539f0f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:40 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:38624
I20260809 04:47:40.577805 22918 tablet_replica.cc:330] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3: stopping tablet replica
I20260809 04:47:40.577807 22750 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cac2be219b64e88bcac09e61539f0f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:40 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:37478
I20260809 04:47:40.578299 22918 raft_consensus.cc:2229] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:40.578454 22610 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cac2be219b64e88bcac09e61539f0f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:40 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:32820
I20260809 04:47:40.578845 22918 raft_consensus.cc:2258] T 0cac2be219b64e88bcac09e61539f0f7 P fd68e45302d3406eb3681420c72792c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:40.579686 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 22359
I20260809 04:47:40.583856 22922 tablet_replica.cc:330] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: stopping tablet replica
I20260809 04:47:40.584205 22922 raft_consensus.cc:2229] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:47:40.584642 22922 raft_consensus.cc:2258] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:40.585037 22921 tablet_replica.cc:330] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: stopping tablet replica
I20260809 04:47:40.585345 22921 raft_consensus.cc:2229] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:40.585672 22921 raft_consensus.cc:2258] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:40.587586 22922 ts_tablet_manager.cc:1905] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:40.588691 22921 ts_tablet_manager.cc:1905] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:40.590927 22922 ts_tablet_manager.cc:1918] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:40.591077 22922 log.cc:1196] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/0cac2be219b64e88bcac09e61539f0f7
I20260809 04:47:40.591519 22922 ts_tablet_manager.cc:1939] T 0cac2be219b64e88bcac09e61539f0f7 P 1a0de947867e4b868fd16a1111d2cc72: Deleting consensus metadata
I20260809 04:47:40.593114 22300 catalog_manager.cc:4841] TS 1a0de947867e4b868fd16a1111d2cc72 (127.7.171.2:42221): tablet 0cac2be219b64e88bcac09e61539f0f7 (table test-table [id=e34143595e2c477d970bff44e150b074]) successfully deleted
I20260809 04:47:40.596119 22921 ts_tablet_manager.cc:1918] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:40.596328 22921 log.cc:1196] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/0cac2be219b64e88bcac09e61539f0f7
W20260809 04:47:40.596395 22303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:34105 (error 108)
I20260809 04:47:40.596802 22921 ts_tablet_manager.cc:1939] T 0cac2be219b64e88bcac09e61539f0f7 P fcc6747e1504414dbb91c6153eeb644c: Deleting consensus metadata
I20260809 04:47:40.597550 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 22541
I20260809 04:47:40.602124 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:40.601740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:34105 (local address 127.0.0.1:45248)
0809 04:47:40.601913 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:40.602007 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:34105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 04:47:40.602402 22303 catalog_manager.cc:4584] TS fd68e45302d3406eb3681420c72792c3 (127.7.171.1:34105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0cac2be219b64e88bcac09e61539f0f7: Network error: Client connection negotiation failed: client connection to 127.7.171.1:34105: connect: Connection refused (error 111)
I20260809 04:47:40.605275 22301 catalog_manager.cc:4841] TS fcc6747e1504414dbb91c6153eeb644c (127.7.171.3:39049): tablet 0cac2be219b64e88bcac09e61539f0f7 (table test-table [id=e34143595e2c477d970bff44e150b074]) successfully deleted
I20260809 04:47:40.609477 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 22696
I20260809 04:47:40.620785 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 22282
I20260809 04:47:40.630964 7852 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 04:47:40.976349 7852 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 04:47:40.976470 7852 mini_ranger.cc:66] Stopping Ranger...
I20260809 04:47:41.321774 7852 mini_ranger.cc:68] Stopped Ranger
2026-08-09 04:47:41.322 UTC [21270] LOG: received smart shutdown request
2026-08-09 04:47:41.326 UTC [21270] LOG: background worker "logical replication launcher" (PID 21280) exited with exit code 1
2026-08-09 04:47:41.326 UTC [21275] LOG: shutting down
2026-08-09 04:47:41.523 UTC [21270] LOG: database system is shut down
2026-08-09T04:47:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51343 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:47:41 dist-test-slave-1zqn krb5kdc[22932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:47:41 dist-test-slave-1zqn krb5kdc[22932](info): set up 2 sockets
Aug 09 04:47:41 dist-test-slave-1zqn krb5kdc[22932](info): commencing operation
krb5kdc: starting...
W20260809 04:47:42.802433 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:47:42 dist-test-slave-1zqn krb5kdc[22932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:47:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:47:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:47:42.939810 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:38453
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:38453,127.7.171.61:35969,127.7.171.60:45591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:43.059898 22946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:43.060302 22946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:43.060354 22946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:43.064756 22946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:43.064818 22946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:43.064838 22946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:43.064854 22946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:43.064870 22946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:43.070163 22946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:38453,127.7.171.61:35969,127.7.171.60:45591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:38453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:43.071254 22946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:43.072340 22946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:43.081259 22952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:43.081276 22951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:43.081334 22946 server_base.cc:1034] running on GCE node
W20260809 04:47:43.081280 22954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:43.082029 22946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:43.083175 22946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:43.084335 22946 hybrid_clock.cc:648] HybridClock initialized: now 1786250863084302 us; error 49 us; skew 500 ppm
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:43.087271 22946 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:47:43.088243 22946 webserver.cc:460] Webserver started at http://127.7.171.62:42845/ using document root <none> and password file <none>
I20260809 04:47:43.088822 22946 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:43.088883 22946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:43.089121 22946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:43.091235 22946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "75f344faf25b4c0e8081804498546e23"
format_stamp: "Formatted at 2026-08-09 04:47:43 on dist-test-slave-1zqn"
server_key: "978596e116dd10b92b277e23a395511e"
server_key_iv: "c35a4a2d3c249ebac38162a7cc19be8d"
server_key_version: "encryptionkey@0"
I20260809 04:47:43.091830 22946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "75f344faf25b4c0e8081804498546e23"
format_stamp: "Formatted at 2026-08-09 04:47:43 on dist-test-slave-1zqn"
server_key: "978596e116dd10b92b277e23a395511e"
server_key_iv: "c35a4a2d3c249ebac38162a7cc19be8d"
server_key_version: "encryptionkey@0"
I20260809 04:47:43.095672 22946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 04:47:43.098309 22961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:43.099592 22946 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:47:43.099725 22946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "75f344faf25b4c0e8081804498546e23"
format_stamp: "Formatted at 2026-08-09 04:47:43 on dist-test-slave-1zqn"
server_key: "978596e116dd10b92b277e23a395511e"
server_key_iv: "c35a4a2d3c249ebac38162a7cc19be8d"
server_key_version: "encryptionkey@0"
I20260809 04:47:43.099833 22946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:43.115988 22946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:43.119179 22946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:43.119385 22946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:43.126986 22946 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:38453
I20260809 04:47:43.126983 23017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:38453 every 8 connection(s)
I20260809 04:47:43.127962 22946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:47:43.130803 23018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:43.134490 23018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:47:43.137065 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 22946
I20260809 04:47:43.137228 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:47:43.137522 7852 external_mini_cluster.cc:1428] Setting key bdafbccb3cf73a93010d540989bf7b34
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.171.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.61@KRBTEST.COM" created.
I20260809 04:47:43.151253 23021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.137589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55800 (local address 127.7.171.62:38453)
0809 04:47:43.137906 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.137921 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.137972 (+ 51us) server_negotiation.cc:407] Connection header received
0809 04:47:43.138535 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.138544 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.138822 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.139033 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.139959 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.140903 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.142421 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.142850 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.143003 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.145941 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.145947 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.145957 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.145978 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.147767 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.148357 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.148368 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.148369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.148514 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.148854 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.148859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.148861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.149260 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.149403 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.149829 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.150128 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":208,"thread_start_us":105,"threads_started":1}
I20260809 04:47:43.151249 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.136696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:38453 (local address 127.0.0.1:55800)
0809 04:47:43.137170 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.137222 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:43.138200 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:43.139040 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:43.139044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:43.139376 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:43.139779 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.139803 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.141037 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.141043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.142292 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.142306 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.143039 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.143045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.143183 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.143807 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:43.143836 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:43.145783 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:43.147879 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.147890 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.147901 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.148223 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.148620 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.148631 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.148633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.148757 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.149386 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:43.149393 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:43.149545 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 04:47:43.149768 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 04:47:43.150173 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":155,"thread_start_us":132,"threads_started":1}
I20260809 04:47:43.154050 23018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:47:43.154872 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.154585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:35969 (local address 127.0.0.1:56772)
0809 04:47:43.154725 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.154807 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:35969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 04:47:43.155208 22963 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.171.61:35969: connect: Connection refused (error 111)
I20260809 04:47:43.156271 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.156005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:35969 (local address 127.0.0.1:56786)
0809 04:47:43.156168 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.156216 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:35969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260809 04:47:43.156509 23018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.61:35969: Network error: Client connection negotiation failed: client connection to 127.7.171.61:35969: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab.
Entry for principal kudu/127.7.171.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab.
I20260809 04:47:43.167640 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.61:35969
--webserver_interface=127.7.171.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab
--principal=kudu/127.7.171.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:38453,127.7.171.61:35969,127.7.171.60:45591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:47:43.205731 23018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } attempt: 1
I20260809 04:47:43.206642 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.206333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:35969 (local address 127.0.0.1:56800)
0809 04:47:43.206485 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.206553 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:35969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 04:47:43.207463 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.207218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:35969 (local address 127.0.0.1:56806)
0809 04:47:43.207350 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.207403 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:35969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 04:47:43.207681 23018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.61:35969: Network error: Client connection negotiation failed: client connection to 127.7.171.61:35969: connect: Connection refused (error 111)
I20260809 04:47:43.275878 23018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } attempt: 2
I20260809 04:47:43.276757 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.276438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:35969 (local address 127.0.0.1:56822)
0809 04:47:43.276600 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.276673 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:35969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 04:47:43.277626 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.277405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:35969 (local address 127.0.0.1:56824)
0809 04:47:43.277522 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.277571 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:35969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 04:47:43.277789 23018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.61:35969: Network error: Client connection negotiation failed: client connection to 127.7.171.61:35969: connect: Connection refused (error 111)
W20260809 04:47:43.291008 23025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:43.291273 23025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:43.291312 23025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:43.295383 23025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:43.295449 23025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:43.295468 23025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:43.295485 23025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:43.295500 23025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:43.300920 23025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:38453,127.7.171.61:35969,127.7.171.60:45591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab
--principal=kudu/127.7.171.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.61:35969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.171.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:43.301990 23025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:43.303050 23025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:43.311971 23031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:43.312033 23033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:43.311970 23030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:43.312270 23025 server_base.cc:1034] running on GCE node
I20260809 04:47:43.312805 23025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:43.313987 23025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:43.315178 23025 hybrid_clock.cc:648] HybridClock initialized: now 1786250863315162 us; error 28 us; skew 500 ppm
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:43.318001 23025 init.cc:377] Logged in from keytab as kudu/127.7.171.61@KRBTEST.COM (short username kudu)
I20260809 04:47:43.318871 23025 webserver.cc:460] Webserver started at http://127.7.171.61:32961/ using document root <none> and password file <none>
I20260809 04:47:43.319404 23025 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:43.319463 23025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:43.319628 23025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:43.321715 23025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/data/instance:
uuid: "bf66cd42c50947b38907aae1bf19d51e"
format_stamp: "Formatted at 2026-08-09 04:47:43 on dist-test-slave-1zqn"
server_key: "f85aaef85dbee10fb049348d71f65621"
server_key_iv: "6ef66176577b1fa614d3fe331bad3f8b"
server_key_version: "encryptionkey@0"
I20260809 04:47:43.322235 23025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/wal/instance:
uuid: "bf66cd42c50947b38907aae1bf19d51e"
format_stamp: "Formatted at 2026-08-09 04:47:43 on dist-test-slave-1zqn"
server_key: "f85aaef85dbee10fb049348d71f65621"
server_key_iv: "6ef66176577b1fa614d3fe331bad3f8b"
server_key_version: "encryptionkey@0"
I20260809 04:47:43.326035 23025 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:47:43.328327 23043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:43.329613 23025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:47:43.329768 23025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/wal
uuid: "bf66cd42c50947b38907aae1bf19d51e"
format_stamp: "Formatted at 2026-08-09 04:47:43 on dist-test-slave-1zqn"
server_key: "f85aaef85dbee10fb049348d71f65621"
server_key_iv: "6ef66176577b1fa614d3fe331bad3f8b"
server_key_version: "encryptionkey@0"
I20260809 04:47:43.329879 23025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:43.350340 23025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:43.354058 23025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:43.354293 23025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:43.362006 23025 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.61:35969
I20260809 04:47:43.362035 23098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.61:35969 every 8 connection(s)
I20260809 04:47:43.362990 23025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
I20260809 04:47:43.365401 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 23025
I20260809 04:47:43.365545 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-1/wal/instance
I20260809 04:47:43.365828 7852 external_mini_cluster.cc:1428] Setting key d27084d27794cb259a631ea75bdc7c0b
I20260809 04:47:43.365983 23100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:43.369016 23018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } attempt: 3
I20260809 04:47:43.371927 23100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.171.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.60@KRBTEST.COM" created.
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:43.386442 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.369653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:35969 (local address 127.0.0.1:56826)
0809 04:47:43.369876 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.369893 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:43.369990 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:43.372570 (+ 2580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:43.372573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:43.372786 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:43.372931 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.372940 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.374983 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.374987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.375760 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.375771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.376617 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.376621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.376722 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.377369 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:43.377391 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:43.381686 (+ 4295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:43.384202 (+ 2516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.384207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.384210 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.384620 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.384965 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.384968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.384970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.385079 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.385908 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:43.385911 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:43.385995 (+ 84us) client_negotiation.cc:769] Sending connection context
0809 04:47:43.386108 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 04:47:43.386263 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260809 04:47:43.387405 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.373922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:38453 (local address 127.0.0.1:55812)
0809 04:47:43.374364 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.374412 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:43.374567 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:43.374881 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:43.374888 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:43.375353 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:43.375502 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.375509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.377148 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.377154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.377921 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.377931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.378476 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.378481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.378595 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.379160 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:43.379176 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:43.382509 (+ 3333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:43.384657 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.384671 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.384684 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.385000 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.385297 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.385300 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.385302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.385386 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.385776 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:43.385784 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:43.385964 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 04:47:43.386144 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 04:47:43.386492 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":138,"threads_started":1}
I20260809 04:47:43.387409 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.370428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.7.171.61:35969)
0809 04:47:43.371048 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.371064 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.371102 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:47:43.371803 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.371823 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.372239 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.372598 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.373104 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.374861 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.375882 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.376475 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.376653 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.381843 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.381850 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.381864 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.381895 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.384018 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.384730 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.384738 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.384740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.384857 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.385161 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.385167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.385171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.385786 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.385919 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.386244 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.386589 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":110,"server-negotiator.queue_time_us":489,"spinlock_wait_cycles":2944,"thread_start_us":165,"threads_started":1}
I20260809 04:47:43.388320 23021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.373957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55812 (local address 127.7.171.62:38453)
0809 04:47:43.374095 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.374100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.374486 (+ 386us) server_negotiation.cc:407] Connection header received
0809 04:47:43.374688 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.374691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.374740 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.374828 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.375648 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.376976 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.378050 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.378320 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.378433 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.382676 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.382680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.382685 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.382719 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.384537 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.385112 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.385119 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.385123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.385199 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.385479 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.385484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.385486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.385667 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.385776 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.388105 (+ 2329us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.388216 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:47:43.389524 23100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:47:43.389648 23018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:47:43.390367 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.390056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49836)
0809 04:47:43.390227 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.390297 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 04:47:43.391264 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.390935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49840)
0809 04:47:43.391133 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.391190 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260809 04:47:43.391410 23018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45591: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
I20260809 04:47:43.398039 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.390056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:35969 (local address 127.0.0.1:56838)
0809 04:47:43.390273 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.390286 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:43.390363 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:43.390815 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:43.390818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:43.390986 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:43.391051 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.391056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.391884 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.391889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.392458 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.392466 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.392918 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.392922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.393000 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.393482 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:43.393493 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:43.394903 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:43.396577 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.396580 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.396581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.396790 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.397042 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.397046 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.397048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.397132 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.397522 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:43.397524 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:43.397623 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 04:47:43.397724 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 04:47:43.397868 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260809 04:47:43.398504 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.390246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56838 (local address 127.7.171.61:35969)
0809 04:47:43.390339 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.390342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.390354 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:47:43.390679 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.390681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.390716 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.390804 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.391158 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.391779 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.392571 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.392812 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.392920 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.395039 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.395043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.395045 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.395065 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.396476 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.396877 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.396879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.396880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.396947 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.397229 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.397232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.397234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.397404 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.397522 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.397779 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.397926 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":35}
I20260809 04:47:43.399284 23100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:47:43.400014 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.399739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49846)
0809 04:47:43.399851 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.399930 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 04:47:43.400386 23047 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
I20260809 04:47:43.401296 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.401013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49848)
0809 04:47:43.401196 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.401242 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 04:47:43.401481 23100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45591: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab.
Entry for principal kudu/127.7.171.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab.
I20260809 04:47:43.402755 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.60:45591
--webserver_interface=127.7.171.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab
--principal=kudu/127.7.171.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:38453,127.7.171.61:35969,127.7.171.60:45591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:47:43.422580 23018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } attempt: 1
I20260809 04:47:43.423468 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.423165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49856)
0809 04:47:43.423317 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.423387 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 04:47:43.424263 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.424021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49864)
0809 04:47:43.424154 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.424202 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 04:47:43.424423 23018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45591: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
I20260809 04:47:43.450822 23100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } attempt: 1
I20260809 04:47:43.451803 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.451485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49876)
0809 04:47:43.451653 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.451721 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 04:47:43.452612 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.452383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49884)
0809 04:47:43.452499 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.452556 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 04:47:43.452756 23100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45591: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
I20260809 04:47:43.499609 23018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } attempt: 2
I20260809 04:47:43.500589 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.500236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49896)
0809 04:47:43.500423 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.500499 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 04:47:43.501526 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.501278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49910)
0809 04:47:43.501399 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.501462 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:47:43.501698 23018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45591: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
W20260809 04:47:43.520048 23108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:43.520403 23108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:43.520514 23108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 04:47:43.520915 23100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } attempt: 2
I20260809 04:47:43.521726 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.521431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49920)
0809 04:47:43.521574 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.521648 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 04:47:43.522437 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.522223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49930)
0809 04:47:43.522330 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.522385 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 04:47:43.522567 23100 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45591: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45591: connect: Connection refused (error 111)
W20260809 04:47:43.524823 23108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:43.524899 23108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:43.524945 23108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:43.524987 23108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:43.525035 23108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:43.530400 23108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:38453,127.7.171.61:35969,127.7.171.60:45591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab
--principal=kudu/127.7.171.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.60:45591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.171.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:43.531486 23108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:43.532604 23108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:43.541427 23116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:43.541487 23114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:43.541471 23113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:43.541535 23108 server_base.cc:1034] running on GCE node
I20260809 04:47:43.542325 23108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:43.543481 23108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:43.544679 23108 hybrid_clock.cc:648] HybridClock initialized: now 1786250863544644 us; error 50 us; skew 500 ppm
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:43.547536 23108 init.cc:377] Logged in from keytab as kudu/127.7.171.60@KRBTEST.COM (short username kudu)
I20260809 04:47:43.548527 23108 webserver.cc:460] Webserver started at http://127.7.171.60:45235/ using document root <none> and password file <none>
I20260809 04:47:43.549105 23108 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:43.549211 23108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:43.549454 23108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:43.551537 23108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/data/instance:
uuid: "9deaf8ee5db540a8b1c3bd43a6c02215"
format_stamp: "Formatted at 2026-08-09 04:47:43 on dist-test-slave-1zqn"
server_key: "cbbef98efb5ad8c219395b24bbe62587"
server_key_iv: "f513ce7aae5ab8fa0c95e8cb5b0763bb"
server_key_version: "encryptionkey@0"
I20260809 04:47:43.552192 23108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/wal/instance:
uuid: "9deaf8ee5db540a8b1c3bd43a6c02215"
format_stamp: "Formatted at 2026-08-09 04:47:43 on dist-test-slave-1zqn"
server_key: "cbbef98efb5ad8c219395b24bbe62587"
server_key_iv: "f513ce7aae5ab8fa0c95e8cb5b0763bb"
server_key_version: "encryptionkey@0"
I20260809 04:47:43.556030 23108 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 04:47:43.558537 23123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:43.559779 23108 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:47:43.559926 23108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/wal
uuid: "9deaf8ee5db540a8b1c3bd43a6c02215"
format_stamp: "Formatted at 2026-08-09 04:47:43 on dist-test-slave-1zqn"
server_key: "cbbef98efb5ad8c219395b24bbe62587"
server_key_iv: "f513ce7aae5ab8fa0c95e8cb5b0763bb"
server_key_version: "encryptionkey@0"
I20260809 04:47:43.560045 23108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:43.579596 23108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:43.582767 23108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:43.582991 23108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:43.590819 23108 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.60:45591
I20260809 04:47:43.590844 23185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.60:45591 every 8 connection(s)
I20260809 04:47:43.591854 23108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
I20260809 04:47:43.595121 23186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:43.599303 23186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:47:43.600478 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 23108
I20260809 04:47:43.600612 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250697618759-7852-0/minicluster-data/master-2/wal/instance
I20260809 04:47:43.600929 7852 external_mini_cluster.cc:1428] Setting key e194d3a4d170f2e83313710e91cc0fad
I20260809 04:47:43.600863 23018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } attempt: 3
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
I20260809 04:47:43.613732 23100 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } attempt: 3
I20260809 04:47:43.616725 23020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.601356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49940)
0809 04:47:43.601957 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.601975 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:43.602083 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:43.604167 (+ 2084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:43.604170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:43.604421 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:43.604550 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.604559 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.606338 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.606342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.607204 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.607215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.608010 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.608017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.608121 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.608727 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:43.608738 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:43.610275 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:43.613458 (+ 3183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.613463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.613466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.613882 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.615020 (+ 1138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.615023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.615025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.615411 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.616013 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:43.616020 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:43.616147 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 04:47:43.616228 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 04:47:43.616362 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"spinlock_wait_cycles":896}
I20260809 04:47:43.617558 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.602058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:38453 (local address 127.0.0.1:55826)
0809 04:47:43.602866 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.602987 (+ 121us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:43.603643 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:43.604154 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:43.604163 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:43.604583 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:43.605342 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.605357 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.606422 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.606427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.607170 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.607178 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.607808 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.607813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.607976 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.608912 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:43.608933 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:43.611450 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:43.614082 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.614094 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.614106 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.614455 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.614768 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.614771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.614773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.614923 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.615531 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:43.615539 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:43.615892 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 04:47:43.616087 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 04:47:43.616436 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":69,"spinlock_wait_cycles":58880,"thread_start_us":200,"threads_started":1}
I20260809 04:47:43.617560 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.602058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.7.171.60:45591)
0809 04:47:43.602701 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.602726 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.602767 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:47:43.603610 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.603634 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.603933 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.605005 (+ 1072us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.605163 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.606320 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.607441 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.607883 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.608397 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.611603 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.611622 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.611625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.611647 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.614357 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.614895 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.614898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.614899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.615004 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.615354 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.615360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.615365 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.615891 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.616719 (+ 828us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.616825 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.616967 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":209,"server-negotiator.queue_time_us":501,"spinlock_wait_cycles":57856,"thread_start_us":156,"threads_started":1}
I20260809 04:47:43.617676 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.602058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49940 (local address 127.7.171.60:45591)
0809 04:47:43.602717 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.602728 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.602767 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:47:43.603609 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.603634 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.603933 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.604732 (+ 799us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.605116 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.606220 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.607351 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.607879 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.607982 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.610438 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.610447 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.610464 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.610498 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.613333 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.614734 (+ 1401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.614742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.614747 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.614903 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.615522 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.615526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.615528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.615887 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.617280 (+ 1393us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.617356 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.617468 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":543,"server-negotiator.queue_time_us":584,"spinlock_wait_cycles":13440,"thread_start_us":87,"threads_started":1}
I20260809 04:47:43.618783 23021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.602249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55826 (local address 127.7.171.62:38453)
0809 04:47:43.602426 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.602430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.603073 (+ 643us) server_negotiation.cc:407] Connection header received
0809 04:47:43.603850 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.603853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.603909 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.604023 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.605501 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.606278 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.607300 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.607641 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.607744 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.611604 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.611608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.611610 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.611637 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.613943 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.614546 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.614553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.614558 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.614651 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.615020 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.615024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.615025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.615411 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.618416 (+ 3005us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.618503 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.618623 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51,"spinlock_wait_cycles":3328}
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
I20260809 04:47:43.619760 23186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:47:43.623646 23018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23: Bootstrap starting.
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
I20260809 04:47:43.625671 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.614259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49958)
0809 04:47:43.614495 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.614512 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:43.614616 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:43.614985 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:43.614988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:43.615186 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:43.615316 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.615325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.616507 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.616511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.617298 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.617309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.617973 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.617977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.618061 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.618561 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:43.618584 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:43.620664 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:43.623807 (+ 3143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.623811 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.623813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.624120 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.624534 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.624537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.624539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.624637 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.625126 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:43.625129 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:43.625242 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 04:47:43.625360 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 04:47:43.625506 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260809 04:47:43.625778 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.614293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49958 (local address 127.7.171.60:45591)
0809 04:47:43.614718 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.614728 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.614741 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:47:43.614774 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.614776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.614826 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.615430 (+ 604us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.615552 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.616373 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.617445 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.617833 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.618274 (+ 441us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.620813 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.620816 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.620819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.620843 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.623665 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.624293 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.624296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.624298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.624422 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.624757 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.624760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.624762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.624982 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.625133 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.625437 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.625553 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":344,"thread_start_us":119,"threads_started":1}
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:43.628123 23018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:43.628957 23100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e: Bootstrap starting.
I20260809 04:47:43.629565 23018 log.cc:824] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:43.630299 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.620275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:35969 (local address 127.0.0.1:56842)
0809 04:47:43.620841 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.620858 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:43.620932 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:43.621216 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:43.621222 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:43.621414 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:43.621534 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.621541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.622468 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.622473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.623041 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.623050 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.623713 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.623716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.623808 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.624279 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:43.624293 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:43.626268 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:43.628402 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.628406 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.628408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.628661 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.628946 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.628950 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.628953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.629116 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.629769 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:43.629774 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:43.629878 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 04:47:43.629988 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 04:47:43.630134 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497}
I20260809 04:47:43.630873 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.620355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.7.171.61:35969)
0809 04:47:43.620678 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.620683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.621011 (+ 328us) server_negotiation.cc:407] Connection header received
0809 04:47:43.621051 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.621054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.621094 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.621173 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.621664 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.622362 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.623177 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.623504 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.623603 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.626446 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.626451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.626455 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.626486 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.628208 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.628764 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.628768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.628770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.628852 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.629237 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.629241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.629244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.629534 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.629645 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.630567 (+ 922us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.630712 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260809 04:47:43.631485 23186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:47:43.632416 23100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:43.632599 23021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.620828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.7.171.62:38453)
0809 04:47:43.621223 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.621227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.621241 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:47:43.621274 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.621276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.621311 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.621424 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.622180 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.622869 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.623943 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.624388 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.624491 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.628248 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.628254 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.628258 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.628290 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.630233 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.630728 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.630731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.630733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.630813 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.631141 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.631144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.631146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.631355 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.631541 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.632319 (+ 778us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.632446 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326}
I20260809 04:47:43.633386 23100 log.cc:824] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:43.633698 23018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23: No bootstrap required, opened a new log
I20260809 04:47:43.635917 23100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e: No bootstrap required, opened a new log
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250863, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
I20260809 04:47:43.638646 23018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } }
I20260809 04:47:43.638981 23018 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:43.639094 23018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f344faf25b4c0e8081804498546e23, State: Initialized, Role: FOLLOWER
I20260809 04:47:43.639685 23018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } }
Aug 09 04:47:43 dist-test-slave-1zqn krb5kdc[22932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
I20260809 04:47:43.640156 23100 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } }
I20260809 04:47:43.640352 23100 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:43.640456 23100 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf66cd42c50947b38907aae1bf19d51e, State: Initialized, Role: FOLLOWER
I20260809 04:47:43.640998 23100 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } }
I20260809 04:47:43.641366 23018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:43.642423 23100 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:43.642660 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.631984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45591 (local address 127.0.0.1:49960)
0809 04:47:43.632130 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:43.632145 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:43.632221 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:43.633331 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:43.633333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:43.633498 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:43.633596 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.633603 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.634742 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.634746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.635538 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:43.635547 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.636129 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.636132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:43.636208 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.636719 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:43.636734 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:43.638242 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:43.640363 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.640369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.640373 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.640625 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.640927 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:43.640930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:43.640932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:43.641017 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:43.642056 (+ 1039us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:43.642064 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:43.642198 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 04:47:43.642300 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 04:47:43.642448 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 04:47:43.643245 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.632076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49960 (local address 127.7.171.60:45591)
0809 04:47:43.633100 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.633105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.633119 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:47:43.633163 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.633169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.633211 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.633715 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.633805 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.634605 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.635667 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.636006 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.637367 (+ 1361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.638382 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.638389 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.638392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.638416 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.640240 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.640747 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.640750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.640752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.640836 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.641119 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.641136 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.641137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.641680 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.642859 (+ 1179us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.642947 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.643063 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":954}
I20260809 04:47:43.644423 23196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } } }
I20260809 04:47:43.644665 23196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:47:43.644887 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.634010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56858 (local address 127.7.171.61:35969)
0809 04:47:43.634117 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.634121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.634144 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:47:43.634267 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.634270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.634306 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.634369 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.635192 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.635895 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.637070 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.637426 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.637526 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.640935 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.640940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.640942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.640971 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.642731 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.643247 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.643253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.643257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.643357 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.643710 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.643714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.643715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.643979 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.644209 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.644636 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.644753 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:47:43.646579 23186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215: Bootstrap starting.
I20260809 04:47:43.648633 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:47:43.649724 23186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:43.650645 23186 log.cc:824] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:43.646895 22946 master_runner.cc:186] Error getting master registration for 127.7.171.61:35969: OK, instance_id { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" instance_seqno: 1786250863359878 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:47:43.651170 22946 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 04:47:43.651299 23215 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:47:43.651379 23215 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:47:43.652987 23186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215: No bootstrap required, opened a new log
W20260809 04:47:43.655022 23224 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:47:43.655164 23224 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:47:43.656216 23197 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } } }
I20260809 04:47:43.656414 23197 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:47:43.658025 23186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } }
I20260809 04:47:43.658339 23186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:43.658432 23186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9deaf8ee5db540a8b1c3bd43a6c02215, State: Initialized, Role: FOLLOWER
I20260809 04:47:43.659003 23186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } }
I20260809 04:47:43.656265 23025 master_runner.cc:186] Error getting master registration for 127.7.171.60:45591: OK, instance_id { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" instance_seqno: 1786250863588625 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:47:43.660220 23025 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:47:43.660470 23186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:47:43.667974 23248 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:47:43.668160 23248 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:47:43.668880 23108 master_runner.cc:263] No leader master found from master 9deaf8ee5db540a8b1c3bd43a6c02215
I20260809 04:47:43.668948 23108 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:47:43.678280 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.657732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56874 (local address 127.7.171.61:35969)
0809 04:47:43.657833 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.657838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.668390 (+ 10552us) server_negotiation.cc:407] Connection header received
0809 04:47:43.668603 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.668606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.668673 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.668731 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.669697 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.670456 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.671615 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.671980 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.672134 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.674040 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.674044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.674047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.674073 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.675854 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.676385 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.676389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.676391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.676487 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.676788 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.676792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.676793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.677013 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.677157 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.677961 (+ 804us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.678089 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 04:47:43.684041 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.656977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49974 (local address 127.7.171.60:45591)
0809 04:47:43.657097 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.657103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.671276 (+ 14173us) server_negotiation.cc:407] Connection header received
0809 04:47:43.671426 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.671429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.671474 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.671525 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.674483 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.675367 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.677530 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.677895 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.677976 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.680432 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.680437 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.680440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.680466 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.682158 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.682652 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.682656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.682658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.682750 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.683054 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.683058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.683060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.683282 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.683410 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.683741 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.683866 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 04:47:43.686707 23234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } } }
I20260809 04:47:43.686945 23234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:47:43.688226 23021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:43.657320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55840 (local address 127.7.171.62:38453)
0809 04:47:43.657432 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:43.657437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:43.669912 (+ 12475us) server_negotiation.cc:407] Connection header received
0809 04:47:43.670079 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:43.670082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:43.670130 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:43.670179 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:43.672500 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.673308 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.676560 (+ 3252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:43.676919 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:43.676996 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:43.684405 (+ 7409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:43.684409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:43.684412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:43.684442 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:43.686235 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.686723 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.686729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.686733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.686823 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:43.687202 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:43.687206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:43.687208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:43.687428 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:43.687551 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:43.687900 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 04:47:43.688023 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 04:47:43.760759 23196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:43.761005 23196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } }
I20260809 04:47:43.762153 23196 leader_election.cc:288] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf66cd42c50947b38907aae1bf19d51e (127.7.171.61:35969), 9deaf8ee5db540a8b1c3bd43a6c02215 (127.7.171.60:45591)
I20260809 04:47:43.763254 23071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f344faf25b4c0e8081804498546e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf66cd42c50947b38907aae1bf19d51e" is_pre_election: true
I20260809 04:47:43.763592 23071 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f344faf25b4c0e8081804498546e23 in term 0.
I20260809 04:47:43.764412 22963 leader_election.cc:302] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f344faf25b4c0e8081804498546e23, bf66cd42c50947b38907aae1bf19d51e; no voters:
I20260809 04:47:43.765077 23196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:47:43.765169 23196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:47:43.765213 23196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:43.766669 23196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } }
I20260809 04:47:43.767629 23071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f344faf25b4c0e8081804498546e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf66cd42c50947b38907aae1bf19d51e"
I20260809 04:47:43.767760 23071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:43.769152 23071 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f344faf25b4c0e8081804498546e23 in term 1.
I20260809 04:47:43.769586 22963 leader_election.cc:302] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f344faf25b4c0e8081804498546e23, bf66cd42c50947b38907aae1bf19d51e; no voters:
I20260809 04:47:43.771077 23153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f344faf25b4c0e8081804498546e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" is_pre_election: true
I20260809 04:47:43.771433 23153 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f344faf25b4c0e8081804498546e23 in term 0.
I20260809 04:47:43.771714 23151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f344faf25b4c0e8081804498546e23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215"
I20260809 04:47:43.771819 23151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:43.773247 23151 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f344faf25b4c0e8081804498546e23 in term 1.
I20260809 04:47:43.775632 23196 leader_election.cc:288] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [CANDIDATE]: Term 1 election: Requested vote from peers bf66cd42c50947b38907aae1bf19d51e (127.7.171.61:35969), 9deaf8ee5db540a8b1c3bd43a6c02215 (127.7.171.60:45591)
I20260809 04:47:43.775872 23196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:47:43.776021 23196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [term 1 LEADER]: Becoming Leader. State: Replica: 75f344faf25b4c0e8081804498546e23, State: Running, Role: LEADER
I20260809 04:47:43.776546 23196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } }
I20260809 04:47:43.779006 23196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75f344faf25b4c0e8081804498546e23. Latest consensus state: current_term: 1 leader_uuid: "75f344faf25b4c0e8081804498546e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } } }
I20260809 04:47:43.779235 23196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:43.779811 23252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:47:43.781064 23252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:47:43.794698 23151 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [term 1 FOLLOWER]: Refusing update from remote peer 75f344faf25b4c0e8081804498546e23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:43.795688 23250 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:43.801160 23234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75f344faf25b4c0e8081804498546e23. Latest consensus state: current_term: 1 leader_uuid: "75f344faf25b4c0e8081804498546e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } } }
I20260809 04:47:43.801355 23234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:47:43.802831 23071 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [term 1 FOLLOWER]: Refusing update from remote peer 75f344faf25b4c0e8081804498546e23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:43.803643 23250 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:43.809011 23197 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75f344faf25b4c0e8081804498546e23. Latest consensus state: current_term: 1 leader_uuid: "75f344faf25b4c0e8081804498546e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } } }
I20260809 04:47:43.809186 23197 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:47:43.817664 23250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75f344faf25b4c0e8081804498546e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } } }
I20260809 04:47:43.817858 23250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:43.819808 23234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75f344faf25b4c0e8081804498546e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } } }
I20260809 04:47:43.819984 23234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:47:43.820674 23253 mvcc.cc:205] Tried to move back new op lower bound from 7316483538073440256 to 7316483538033549312. Current Snapshot: MvccSnapshot[applied={T|T < 7316483538073440256}]
I20260809 04:47:43.822908 23254 mvcc.cc:205] Tried to move back new op lower bound from 7316483538073440256 to 7316483538033549312. Current Snapshot: MvccSnapshot[applied={T|T < 7316483538073440256}]
I20260809 04:47:43.833035 23196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75f344faf25b4c0e8081804498546e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } } }
I20260809 04:47:43.833226 23196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:43.835000 23197 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75f344faf25b4c0e8081804498546e23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f344faf25b4c0e8081804498546e23" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 38453 } } peers { permanent_uuid: "bf66cd42c50947b38907aae1bf19d51e" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 35969 } } peers { permanent_uuid: "9deaf8ee5db540a8b1c3bd43a6c02215" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45591 } } }
I20260809 04:47:43.835171 23197 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:47:43.837323 23252 catalog_manager.cc:1347] Generated new cluster ID: b09244a288b54d149e17a4e9bccd3448
I20260809 04:47:43.837447 23252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:47:43.856575 23256 mvcc.cc:205] Tried to move back new op lower bound from 7316483538073440256 to 7316483538033549312. Current Snapshot: MvccSnapshot[applied={T|T < 7316483538073440256}]
I20260809 04:47:43.870069 23252 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:47:43.871169 23252 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:47:43.887786 23252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75f344faf25b4c0e8081804498546e23: Generated new TSK 0
I20260809 04:47:43.888859 23252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:47:44.656620 23224 catalog_manager.cc:1259] Loaded cluster ID: b09244a288b54d149e17a4e9bccd3448
I20260809 04:47:44.656713 23224 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e: loading cluster ID for follower catalog manager: success
I20260809 04:47:44.658468 23224 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e: acquiring CA information for follower catalog manager: success
I20260809 04:47:44.659675 23224 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bf66cd42c50947b38907aae1bf19d51e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:47:44.669406 23248 catalog_manager.cc:1259] Loaded cluster ID: b09244a288b54d149e17a4e9bccd3448
I20260809 04:47:44.669477 23248 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215: loading cluster ID for follower catalog manager: success
I20260809 04:47:44.671008 23248 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215: acquiring CA information for follower catalog manager: success
I20260809 04:47:44.672149 23248 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9deaf8ee5db540a8b1c3bd43a6c02215: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:47:44.960752 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 22946
I20260809 04:47:44.970266 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 23025
I20260809 04:47:44.979704 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 23108
2026-08-09T04:47:44Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3332 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:47:45 dist-test-slave-1zqn krb5kdc[23268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:47:45 dist-test-slave-1zqn krb5kdc[23268](info): set up 2 sockets
Aug 09 04:47:45 dist-test-slave-1zqn krb5kdc[23268](info): commencing operation
krb5kdc: starting...
W20260809 04:47:46.084944 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:47:46 dist-test-slave-1zqn krb5kdc[23268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:47:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:47:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:47:46.219887 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:39151
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:39151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:46.335765 23281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:46.336099 23281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:46.336179 23281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:46.340154 23281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:46.340240 23281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:46.340277 23281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:46.340328 23281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:46.340364 23281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:46.345520 23281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:39151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:39151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:46.346541 23281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:46.347548 23281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:46.356122 23287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:46.356101 23290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:46.356294 23281 server_base.cc:1034] running on GCE node
W20260809 04:47:46.356474 23288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:46.357009 23281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:46.358197 23281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:46.359392 23281 hybrid_clock.cc:648] HybridClock initialized: now 1786250866359379 us; error 25 us; skew 500 ppm
Aug 09 04:47:46 dist-test-slave-1zqn krb5kdc[23268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250866, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:46.362226 23281 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:47:46.363072 23281 webserver.cc:460] Webserver started at http://127.7.171.62:35079/ using document root <none> and password file <none>
I20260809 04:47:46.363727 23281 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:46.363840 23281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:46.364199 23281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:46.366355 23281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "fe3b0803d7384d2491c3eed9c8d303bd"
format_stamp: "Formatted at 2026-08-09 04:47:46 on dist-test-slave-1zqn"
server_key: "0c6e82f792faf8192a22e6959738f279"
server_key_iv: "5f4e9699632f17a7cba655700283eefe"
server_key_version: "encryptionkey@0"
I20260809 04:47:46.366935 23281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "fe3b0803d7384d2491c3eed9c8d303bd"
format_stamp: "Formatted at 2026-08-09 04:47:46 on dist-test-slave-1zqn"
server_key: "0c6e82f792faf8192a22e6959738f279"
server_key_iv: "5f4e9699632f17a7cba655700283eefe"
server_key_version: "encryptionkey@0"
I20260809 04:47:46.370878 23281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:47:46.373463 23297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:46.374709 23281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:47:46.374876 23281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "fe3b0803d7384d2491c3eed9c8d303bd"
format_stamp: "Formatted at 2026-08-09 04:47:46 on dist-test-slave-1zqn"
server_key: "0c6e82f792faf8192a22e6959738f279"
server_key_iv: "5f4e9699632f17a7cba655700283eefe"
server_key_version: "encryptionkey@0"
I20260809 04:47:46.375002 23281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:46.385720 23281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:46.388614 23281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:46.388821 23281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:46.396037 23281 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:39151
I20260809 04:47:46.396129 23352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:39151 every 8 connection(s)
I20260809 04:47:46.397099 23281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:47:46.399912 23353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:46.405432 23353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd: Bootstrap starting.
I20260809 04:47:46.407346 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 23281
I20260809 04:47:46.407445 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:47:46.407744 7852 external_mini_cluster.cc:1428] Setting key 2644a8ddb8d0d2330008ccbfbd12d853
I20260809 04:47:46.407995 23353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:46.408841 23353 log.cc:824] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:46.410723 23353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd: No bootstrap required, opened a new log
I20260809 04:47:46.414193 23353 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3b0803d7384d2491c3eed9c8d303bd" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39151 } }
I20260809 04:47:46.414433 23353 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:46.414789 23353 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3b0803d7384d2491c3eed9c8d303bd, State: Initialized, Role: FOLLOWER
I20260809 04:47:46.415282 23353 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3b0803d7384d2491c3eed9c8d303bd" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39151 } }
I20260809 04:47:46.415437 23353 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:47:46.415526 23353 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:47:46.415657 23353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [term 0 FOLLOWER]: Advancing to term 1
Aug 09 04:47:46 dist-test-slave-1zqn krb5kdc[23268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:46.416926 23353 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3b0803d7384d2491c3eed9c8d303bd" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39151 } }
I20260809 04:47:46.417289 23353 leader_election.cc:302] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe3b0803d7384d2491c3eed9c8d303bd; no voters:
I20260809 04:47:46.417649 23353 leader_election.cc:288] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:47:46.417810 23359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:47:46.418399 23359 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [term 1 LEADER]: Becoming Leader. State: Replica: fe3b0803d7384d2491c3eed9c8d303bd, State: Running, Role: LEADER
I20260809 04:47:46.418794 23359 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3b0803d7384d2491c3eed9c8d303bd" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39151 } }
I20260809 04:47:46.419101 23353 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:46.420621 23360 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe3b0803d7384d2491c3eed9c8d303bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3b0803d7384d2491c3eed9c8d303bd" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39151 } } }
I20260809 04:47:46.420745 23360 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:46.420687 23361 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe3b0803d7384d2491c3eed9c8d303bd. Latest consensus state: current_term: 1 leader_uuid: "fe3b0803d7384d2491c3eed9c8d303bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3b0803d7384d2491c3eed9c8d303bd" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39151 } } }
I20260809 04:47:46.420801 23361 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:46.421108 23370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:47:46.423070 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.408855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44440 (local address 127.7.171.62:39151)
0809 04:47:46.409323 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.409343 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.409389 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:47:46.410091 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.410109 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.410428 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.410772 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:46.411594 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.412695 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.413564 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.414054 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.414230 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.417166 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:46.417172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:46.417185 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:46.417225 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:46.419445 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.419990 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.419995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.420002 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.420218 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.420511 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.420517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.420521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.421084 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:46.421318 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.421776 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.422087 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":132,"threads_started":1}
I20260809 04:47:46.425637 23370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:47:46.426190 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:47:46.431674 23370 catalog_manager.cc:1347] Generated new cluster ID: 5de852de245b4644b652613f1ba27162
I20260809 04:47:46.431747 23370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:47:46.438961 23370 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:47:46.439886 23370 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:47:46.451409 23370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe3b0803d7384d2491c3eed9c8d303bd: Generated new TSK 0
I20260809 04:47:46.452198 23370 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:47:46.482731 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 23281
2026-08-09T04:47:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1502 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:47:46 dist-test-slave-1zqn krb5kdc[23384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:47:46 dist-test-slave-1zqn krb5kdc[23384](info): set up 2 sockets
Aug 09 04:47:46 dist-test-slave-1zqn krb5kdc[23384](info): commencing operation
krb5kdc: starting...
W20260809 04:47:47.603336 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:47:47 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:47:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:47:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:47:47.734686 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34679
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:34679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:47.849768 23398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:47.850111 23398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:47.850193 23398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:47.854351 23398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:47.854468 23398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:47.854508 23398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:47.854547 23398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:47.854580 23398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:47.860200 23398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:34679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:47.861445 23398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:47.862563 23398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:47.871520 23403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:47.871520 23406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:47.871871 23398 server_base.cc:1034] running on GCE node
W20260809 04:47:47.871524 23404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:47.872473 23398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:47.873596 23398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:47.874763 23398 hybrid_clock.cc:648] HybridClock initialized: now 1786250867874731 us; error 45 us; skew 500 ppm
Aug 09 04:47:47 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250867, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:47.882056 23398 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:47:47.883104 23398 webserver.cc:460] Webserver started at http://127.7.171.62:40161/ using document root <none> and password file <none>
I20260809 04:47:47.883699 23398 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:47.883764 23398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:47.884009 23398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:47.886171 23398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "8cfb760004634a788d6176f118079317"
format_stamp: "Formatted at 2026-08-09 04:47:47 on dist-test-slave-1zqn"
server_key: "4c9c43a7dbd891d424f563e552c5a607"
server_key_iv: "109b1ea157d5f89d59c6b7afd244155b"
server_key_version: "encryptionkey@0"
I20260809 04:47:47.886782 23398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "8cfb760004634a788d6176f118079317"
format_stamp: "Formatted at 2026-08-09 04:47:47 on dist-test-slave-1zqn"
server_key: "4c9c43a7dbd891d424f563e552c5a607"
server_key_iv: "109b1ea157d5f89d59c6b7afd244155b"
server_key_version: "encryptionkey@0"
I20260809 04:47:47.890760 23398 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:47:47.893277 23413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:47.894456 23398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:47:47.894588 23398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "8cfb760004634a788d6176f118079317"
format_stamp: "Formatted at 2026-08-09 04:47:47 on dist-test-slave-1zqn"
server_key: "4c9c43a7dbd891d424f563e552c5a607"
server_key_iv: "109b1ea157d5f89d59c6b7afd244155b"
server_key_version: "encryptionkey@0"
I20260809 04:47:47.894709 23398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:47.920182 23398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:47.923527 23398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:47.923750 23398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:47.931090 23398 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:34679
I20260809 04:47:47.931113 23468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:34679 every 8 connection(s)
I20260809 04:47:47.932156 23398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:47:47.934897 23469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:47.940435 23469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Bootstrap starting.
I20260809 04:47:47.941855 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 23398
I20260809 04:47:47.941951 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:47:47.942234 7852 external_mini_cluster.cc:1428] Setting key 66b6698df1f2bbfe0edf49cf78ef8c2d
I20260809 04:47:47.943048 23469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:47.943944 23469 log.cc:824] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:47.946158 23469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: No bootstrap required, opened a new log
I20260809 04:47:47.949493 23469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:47.949779 23469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:47.949847 23469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cfb760004634a788d6176f118079317, State: Initialized, Role: FOLLOWER
I20260809 04:47:47.950703 23469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:47.950894 23469 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:47:47.950974 23469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:47:47.951104 23469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 04:47:47 dist-test-slave-1zqn krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:47.952064 23469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:47.952445 23469 leader_election.cc:302] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cfb760004634a788d6176f118079317; no voters:
I20260809 04:47:47.952785 23469 leader_election.cc:288] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:47:47.952858 23475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:47:47.953100 23475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 1 LEADER]: Becoming Leader. State: Replica: 8cfb760004634a788d6176f118079317, State: Running, Role: LEADER
I20260809 04:47:47.953488 23475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:47.954646 23469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:47.954716 23475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cfb760004634a788d6176f118079317. Latest consensus state: current_term: 1 leader_uuid: "8cfb760004634a788d6176f118079317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } } }
I20260809 04:47:47.954808 23475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:47.955135 23476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cfb760004634a788d6176f118079317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } } }
I20260809 04:47:47.955247 23476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:47.958530 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:47.943375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39208 (local address 127.7.171.62:34679)
0809 04:47:47.943795 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:47.943812 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:47.943846 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:47:47.944556 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:47.944574 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:47.944897 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:47.945180 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:47.946096 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:47.947139 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:47.948432 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:47.949512 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:47.949642 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:47.952658 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:47.952664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:47.952678 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:47.952705 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:47.954656 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:47.955281 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:47.955287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:47.955293 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:47.955440 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:47.955768 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:47.955771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:47.955773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:47.956353 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:47.956609 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:47.957309 (+ 700us) server_negotiation.cc:299] Negotiation successful
0809 04:47:47.957619 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":151,"threads_started":1}
W20260809 04:47:47.960995 23490 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:47:47.961078 23490 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:47:47.961149 23479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:47:47.962044 23479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:47:47.967803 23479 catalog_manager.cc:1347] Generated new cluster ID: f7da400cef5746219880940dd9cf427a
I20260809 04:47:47.967882 23479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:47:47.980901 23479 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:47:47.981818 23479 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:47:47.987741 23479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Generated new TSK 0
I20260809 04:47:47.988463 23479 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:47:48.027554 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34679
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:48.149549 23496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:48.149808 23496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:48.149852 23496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:48.153751 23496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:48.153805 23496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:48.153860 23496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:47:48.160208 23496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:48.161321 23496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:48.162422 23496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:48.172793 23501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:48.172936 23504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:48.172936 23502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:48.173230 23496 server_base.cc:1034] running on GCE node
I20260809 04:47:48.173658 23496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:48.174340 23496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:48.175544 23496 hybrid_clock.cc:648] HybridClock initialized: now 1786250868175528 us; error 28 us; skew 500 ppm
Aug 09 04:47:48 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250868, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:48.178447 23496 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:47:48.179319 23496 webserver.cc:460] Webserver started at http://127.7.171.1:34103/ using document root <none> and password file <none>
I20260809 04:47:48.179877 23496 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:48.179939 23496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:48.180222 23496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:48.182225 23496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "9ef5af632ba244d98bbea049cb2cdf31"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "bb8ce3a2c2de02247a13ff72502a9278"
server_key_iv: "c3741a5835ecd2ef58dfcca74235ee8c"
server_key_version: "encryptionkey@0"
I20260809 04:47:48.182801 23496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "9ef5af632ba244d98bbea049cb2cdf31"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "bb8ce3a2c2de02247a13ff72502a9278"
server_key_iv: "c3741a5835ecd2ef58dfcca74235ee8c"
server_key_version: "encryptionkey@0"
I20260809 04:47:48.186558 23496 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:47:48.189126 23511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:48.190366 23496 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:47:48.190538 23496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "9ef5af632ba244d98bbea049cb2cdf31"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "bb8ce3a2c2de02247a13ff72502a9278"
server_key_iv: "c3741a5835ecd2ef58dfcca74235ee8c"
server_key_version: "encryptionkey@0"
I20260809 04:47:48.190634 23496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:48.208166 23496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:48.211195 23496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:48.211416 23496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:48.212044 23496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:48.212983 23496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:48.213033 23496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:48.213106 23496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:48.213137 23496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:48.222668 23496 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:37385
I20260809 04:47:48.222697 23634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:37385 every 8 connection(s)
I20260809 04:47:48.223706 23496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:47:48.225960 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 23496
I20260809 04:47:48.226090 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:47:48.226440 7852 external_mini_cluster.cc:1428] Setting key 91a6c988e8f4280e5039d5587a00b852
Aug 09 04:47:48 dist-test-slave-1zqn krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250868, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:47:48.238221 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.225594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:60615 (local address 127.7.171.62:34679)
0809 04:47:48.225792 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:48.225795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:48.226787 (+ 992us) server_negotiation.cc:407] Connection header received
0809 04:47:48.228369 (+ 1582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:48.228372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:48.228428 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:48.228511 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:48.229716 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.230254 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.231432 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.231680 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.231795 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.234342 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:48.234345 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:48.234348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:48.234370 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:48.235908 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:48.236518 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:48.236522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:48.236524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:48.236611 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:48.236973 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:48.236977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:48.236979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:48.237172 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:48.237281 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:48.237908 (+ 627us) server_negotiation.cc:299] Negotiation successful
0809 04:47:48.238034 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 04:47:48.239243 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.226074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:60615)
0809 04:47:48.226642 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:48.226687 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:48.228165 (+ 1478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:48.228700 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:48.228707 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:48.229074 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:48.229566 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.229578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.230412 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.230416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.231300 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.231309 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.231837 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.231842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.232018 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.232670 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:48.232686 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:48.234157 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:48.236043 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:48.236049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:48.236080 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:48.236413 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:48.236709 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:48.236713 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:48.236715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:48.236865 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:48.237283 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:48.237290 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:48.237661 (+ 371us) client_negotiation.cc:769] Sending connection context
0809 04:47:48.237851 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 04:47:48.238177 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":160,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
I20260809 04:47:48.240653 23635 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34679
I20260809 04:47:48.240924 23635 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:48.241540 23635 heartbeater.cc:511] Master 127.7.171.62:34679 requested a full tablet report, sending...
I20260809 04:47:48.243191 23431 ts_manager.cc:194] Registered new tserver with Master: 9ef5af632ba244d98bbea049cb2cdf31 (127.7.171.1:37385)
I20260809 04:47:48.244947 23431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:60615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:47:48.260730 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34679
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:48.374738 23640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:48.374997 23640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:48.375041 23640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:48.379047 23640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:48.379108 23640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:48.379166 23640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:47:48.384470 23640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:48.385533 23640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:48.386579 23640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:48.396641 23645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:48.396734 23640 server_base.cc:1034] running on GCE node
W20260809 04:47:48.396641 23648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:48.396713 23646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:48.397374 23640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:48.398108 23640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:48.399349 23640 hybrid_clock.cc:648] HybridClock initialized: now 1786250868399309 us; error 47 us; skew 500 ppm
Aug 09 04:47:48 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250868, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:48.402256 23640 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:47:48.403141 23640 webserver.cc:460] Webserver started at http://127.7.171.2:38863/ using document root <none> and password file <none>
I20260809 04:47:48.403666 23640 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:48.403723 23640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:48.403885 23640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:48.405928 23640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "98bcb0a3fa254657b2179f217677d759"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "81c244db71210b1446ed99d39716310f"
server_key_iv: "0cf8c9eb61aa10df473214b6d9d179fb"
server_key_version: "encryptionkey@0"
I20260809 04:47:48.406428 23640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "98bcb0a3fa254657b2179f217677d759"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "81c244db71210b1446ed99d39716310f"
server_key_iv: "0cf8c9eb61aa10df473214b6d9d179fb"
server_key_version: "encryptionkey@0"
I20260809 04:47:48.410095 23640 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:47:48.412382 23656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:48.413618 23640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:47:48.413713 23640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "98bcb0a3fa254657b2179f217677d759"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "81c244db71210b1446ed99d39716310f"
server_key_iv: "0cf8c9eb61aa10df473214b6d9d179fb"
server_key_version: "encryptionkey@0"
I20260809 04:47:48.413794 23640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:48.430104 23640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:48.433493 23640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:48.433665 23640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:48.434274 23640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:48.435111 23640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:48.435158 23640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:48.435199 23640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:48.435220 23640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:48.444169 23640 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:35825
I20260809 04:47:48.444192 23780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:35825 every 8 connection(s)
I20260809 04:47:48.445094 23640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:47:48.447923 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 23640
I20260809 04:47:48.448073 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:47:48.448405 7852 external_mini_cluster.cc:1428] Setting key abe86ef15b0b213e6cc7b3f9bd3c1b25
Aug 09 04:47:48 dist-test-slave-1zqn krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250868, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:47:48.461179 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.446900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:42831 (local address 127.7.171.62:34679)
0809 04:47:48.447044 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:48.447048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:48.447966 (+ 918us) server_negotiation.cc:407] Connection header received
0809 04:47:48.449561 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:48.449564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:48.449609 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:48.449681 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:48.451266 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.451791 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.453196 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.453430 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.453540 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.456624 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:48.456631 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:48.456637 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:48.456672 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:48.458903 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:48.459472 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:48.459479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:48.459483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:48.459588 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:48.459931 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:48.459937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:48.459941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:48.460226 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:48.460347 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:48.460857 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 04:47:48.460996 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:47:48.462044 23783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.447275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:42831)
0809 04:47:48.447775 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:48.447835 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:48.449349 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:48.449894 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:48.449901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:48.450386 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:48.451096 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.451121 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.451921 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.451927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.453054 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.453064 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.453595 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.453599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.453773 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.454334 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:48.454360 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:48.456406 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:48.459075 (+ 2669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:48.459079 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:48.459091 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:48.459350 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:48.459689 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:48.459691 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:48.459692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:48.459822 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:48.460335 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:48.460340 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:48.460570 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 04:47:48.460741 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 04:47:48.461055 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":120,"threads_started":1}
I20260809 04:47:48.463246 23781 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34679
I20260809 04:47:48.463536 23781 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
I20260809 04:47:48.464118 23781 heartbeater.cc:511] Master 127.7.171.62:34679 requested a full tablet report, sending...
I20260809 04:47:48.465263 23431 ts_manager.cc:194] Registered new tserver with Master: 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825)
I20260809 04:47:48.466094 23431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:42831
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:47:48.480984 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34679
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:48.594744 23786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:48.594992 23786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:48.595036 23786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:48.599390 23786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:48.599447 23786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:48.599545 23786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:47:48.604863 23786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:48.605937 23786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:48.606972 23786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:48.617372 23791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:48.617552 23786 server_base.cc:1034] running on GCE node
W20260809 04:47:48.617559 23794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:48.617444 23792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:48.618084 23786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:48.618755 23786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:48.619946 23786 hybrid_clock.cc:648] HybridClock initialized: now 1786250868619950 us; error 49 us; skew 500 ppm
Aug 09 04:47:48 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250868, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:48.622906 23786 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:47:48.623889 23786 webserver.cc:460] Webserver started at http://127.7.171.3:39051/ using document root <none> and password file <none>
I20260809 04:47:48.624532 23786 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:48.624598 23786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:48.624846 23786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:48.627020 23786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "2a4acd726da84c088082e33d3af47c87"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "3fff5f4ba32a4bfdc06e0c6a6aec2a37"
server_key_iv: "7dd2c4eb21cf2dd51d41f1e630a21a51"
server_key_version: "encryptionkey@0"
I20260809 04:47:48.627604 23786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "2a4acd726da84c088082e33d3af47c87"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "3fff5f4ba32a4bfdc06e0c6a6aec2a37"
server_key_iv: "7dd2c4eb21cf2dd51d41f1e630a21a51"
server_key_version: "encryptionkey@0"
I20260809 04:47:48.631397 23786 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:47:48.633836 23802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:48.635043 23786 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:47:48.635182 23786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "2a4acd726da84c088082e33d3af47c87"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "3fff5f4ba32a4bfdc06e0c6a6aec2a37"
server_key_iv: "7dd2c4eb21cf2dd51d41f1e630a21a51"
server_key_version: "encryptionkey@0"
I20260809 04:47:48.635288 23786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:48.654088 23786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:48.657706 23786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:48.657922 23786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:48.658569 23786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:48.659446 23786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:48.659495 23786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:48.659564 23786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:48.659603 23786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:48.668340 23786 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:33131
I20260809 04:47:48.668401 23927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:33131 every 8 connection(s)
I20260809 04:47:48.669294 23786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:47:48.678469 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 23786
I20260809 04:47:48.678570 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:47:48.678885 7852 external_mini_cluster.cc:1428] Setting key 15d57561890061d7ea44264040c6001d
Aug 09 04:47:48 dist-test-slave-1zqn krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250868, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:48.683388 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.671110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:57349 (local address 127.7.171.62:34679)
0809 04:47:48.671247 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:48.671251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:48.672106 (+ 855us) server_negotiation.cc:407] Connection header received
0809 04:47:48.673509 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:48.673512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:48.673571 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:48.673689 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:48.674892 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.675436 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.676678 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.676905 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.677016 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.679634 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:48.679638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:48.679640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:48.679674 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:48.681300 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:48.681825 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:48.681828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:48.681830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:48.681913 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:48.682264 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:48.682268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:48.682270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:48.682458 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:48.682568 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:48.683082 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 04:47:48.683212 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:47:48.684296 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.671488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:57349)
0809 04:47:48.671928 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:48.671970 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:48.673244 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:48.673887 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:48.673894 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:48.674266 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:48.674741 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.674755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.675566 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.675569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.676539 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.676548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.677072 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.677076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.677243 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.677807 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:48.677824 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:48.679469 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:48.681418 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:48.681424 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:48.681439 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:48.681710 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:48.682005 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:48.682009 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:48.682012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:48.682163 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:48.682595 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:48.682602 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:48.682848 (+ 246us) client_negotiation.cc:769] Sending connection context
0809 04:47:48.683040 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 04:47:48.683377 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":115,"threads_started":1}
I20260809 04:47:48.685431 23928 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34679
I20260809 04:47:48.685694 23928 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:48.686182 23928 heartbeater.cc:511] Master 127.7.171.62:34679 requested a full tablet report, sending...
I20260809 04:47:48.687201 23431 ts_manager.cc:194] Registered new tserver with Master: 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131)
I20260809 04:47:48.687767 23431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:57349
I20260809 04:47:48.692819 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:47:48.703033 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.694997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39212 (local address 127.7.171.62:34679)
0809 04:47:48.695178 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:48.695188 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:48.695232 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:47:48.695441 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:48.695445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:48.695506 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:48.695613 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:48.696657 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.697255 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.698259 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.698547 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.698699 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.700115 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:48.700119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:48.700121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:48.700152 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:48.701327 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:48.701759 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:48.701763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:48.701765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:48.701848 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:48.702136 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:48.702140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:48.702142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:48.702359 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:48.702504 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:48.702752 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 04:47:48.702871 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 04:47:48.705940 23431 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:47:48.708981 23431 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:47:48.723116 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.717150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.7.171.1:37385)
0809 04:47:48.717460 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:48.717500 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:48.717530 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:47:48.717697 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:48.717705 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:48.717924 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:48.718496 (+ 572us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:48.718678 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.719866 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.721542 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.722413 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.722523 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.722696 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:48.722843 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 04:47:48.722949 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":136,"threads_started":1}
I20260809 04:47:48.723105 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.716990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:37385 (local address 127.0.0.1:53766)
0809 04:47:48.717328 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:48.717374 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:48.717523 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:48.718125 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:48.718134 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:48.718165 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:48.718325 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.718333 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.720011 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.720014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.721407 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.721416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.722520 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.722524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.722641 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.722704 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 04:47:48.722811 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 04:47:48.722949 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":31,"thread_start_us":104,"threads_started":1}
I20260809 04:47:48.723109 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.717737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:33131 (local address 127.0.0.1:59836)
0809 04:47:48.718014 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:48.718029 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:48.718113 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:48.718917 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:48.718920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:48.718938 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:48.719023 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.719029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.720478 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.720482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.721474 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.721483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.722567 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.722571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.722706 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.722738 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:47:48.722804 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 04:47:48.722951 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":72,"thread_start_us":114,"threads_started":1}
I20260809 04:47:48.723749 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.717944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59836 (local address 127.7.171.3:33131)
0809 04:47:48.718440 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:48.718452 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:48.718474 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:47:48.718547 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:48.718552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:48.718757 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:48.719186 (+ 429us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:48.719335 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.720347 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.721627 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.722441 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.723150 (+ 709us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.723430 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:48.723483 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 04:47:48.723595 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":175,"threads_started":1}
I20260809 04:47:48.723945 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.718153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:35825 (local address 127.0.0.1:37008)
0809 04:47:48.718404 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:48.718419 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:48.718507 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:48.719441 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:48.719444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:48.719462 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:48.719569 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.719576 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.721185 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.721188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.722294 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.722305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.723565 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.723568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.723661 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.723690 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:47:48.723754 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 04:47:48.723830 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":116,"threads_started":1}
I20260809 04:47:48.724829 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.718601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.7.171.2:35825)
0809 04:47:48.718968 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:48.718980 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:48.719003 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:47:48.719085 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:48.719093 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:48.719285 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:48.719739 (+ 454us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:48.719878 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.721034 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.722442 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.723456 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.723689 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.723876 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:48.724596 (+ 720us) server_negotiation.cc:299] Negotiation successful
0809 04:47:48.724692 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":153,"threads_started":1}
I20260809 04:47:48.726634 23854 tablet_service.cc:1467] Processing CreateTablet for tablet f747fa95d46648a68b40783ea7c0e537 (DEFAULT_TABLE table=test-table [id=1cde2e9c7b154232a600d1660a0b1a1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:48.726644 23563 tablet_service.cc:1467] Processing CreateTablet for tablet f747fa95d46648a68b40783ea7c0e537 (DEFAULT_TABLE table=test-table [id=1cde2e9c7b154232a600d1660a0b1a1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:48.727619 23854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f747fa95d46648a68b40783ea7c0e537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:48.727687 23563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f747fa95d46648a68b40783ea7c0e537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:48.727371 23709 tablet_service.cc:1467] Processing CreateTablet for tablet f747fa95d46648a68b40783ea7c0e537 (DEFAULT_TABLE table=test-table [id=1cde2e9c7b154232a600d1660a0b1a1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:48.728356 23709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f747fa95d46648a68b40783ea7c0e537. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:48.733805 23949 tablet_bootstrap.cc:492] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87: Bootstrap starting.
I20260809 04:47:48.735771 23951 tablet_bootstrap.cc:492] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759: Bootstrap starting.
I20260809 04:47:48.736407 23950 tablet_bootstrap.cc:492] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Bootstrap starting.
I20260809 04:47:48.736784 23949 tablet_bootstrap.cc:654] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:48.737500 23949 log.cc:824] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:48.738432 23951 tablet_bootstrap.cc:654] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:48.738600 23950 tablet_bootstrap.cc:654] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:48.739727 23949 tablet_bootstrap.cc:492] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87: No bootstrap required, opened a new log
I20260809 04:47:48.739519 23951 log.cc:824] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:48.739907 23950 log.cc:824] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:48.739959 23949 ts_tablet_manager.cc:1397] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:47:48.741667 23951 tablet_bootstrap.cc:492] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759: No bootstrap required, opened a new log
I20260809 04:47:48.741851 23951 ts_tablet_manager.cc:1397] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:47:48.741981 23950 tablet_bootstrap.cc:492] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: No bootstrap required, opened a new log
I20260809 04:47:48.742151 23950 ts_tablet_manager.cc:1397] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:47:48.743364 23949 raft_consensus.cc:348] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } }
I20260809 04:47:48.743618 23949 raft_consensus.cc:374] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:48.743695 23949 raft_consensus.cc:729] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4acd726da84c088082e33d3af47c87, State: Initialized, Role: FOLLOWER
I20260809 04:47:48.744200 23949 consensus_queue.cc:260] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } }
I20260809 04:47:48.744877 23928 heartbeater.cc:503] Master 127.7.171.62:34679 was elected leader, sending a full tablet report...
I20260809 04:47:48.745219 23949 ts_tablet_manager.cc:1428] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 04:47:48.745649 23951 raft_consensus.cc:348] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } }
I20260809 04:47:48.745702 23950 raft_consensus.cc:348] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } }
I20260809 04:47:48.745929 23950 raft_consensus.cc:374] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:48.745992 23951 raft_consensus.cc:374] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:48.746022 23950 raft_consensus.cc:729] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef5af632ba244d98bbea049cb2cdf31, State: Initialized, Role: FOLLOWER
I20260809 04:47:48.746081 23951 raft_consensus.cc:729] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98bcb0a3fa254657b2179f217677d759, State: Initialized, Role: FOLLOWER
I20260809 04:47:48.746496 23950 consensus_queue.cc:260] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } }
I20260809 04:47:48.746496 23951 consensus_queue.cc:260] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } }
I20260809 04:47:48.747368 23781 heartbeater.cc:503] Master 127.7.171.62:34679 was elected leader, sending a full tablet report...
I20260809 04:47:48.747455 23951 ts_tablet_manager.cc:1428] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:47:48.747860 23950 ts_tablet_manager.cc:1428] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:47:48.748761 23635 heartbeater.cc:503] Master 127.7.171.62:34679 was elected leader, sending a full tablet report...
I20260809 04:47:48.774034 23957 raft_consensus.cc:482] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:48.774303 23957 raft_consensus.cc:504] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } }
I20260809 04:47:48.775476 23957 leader_election.cc:288] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131), 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825)
I20260809 04:47:48.780547 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.776468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:43169 (local address 127.7.171.2:35825)
0809 04:47:48.776604 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:48.776608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:48.776702 (+ 94us) server_negotiation.cc:407] Connection header received
0809 04:47:48.776795 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:48.776797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:48.776859 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:48.776948 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:48.777176 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.777880 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.779333 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.779885 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.780033 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.780159 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:48.780331 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 04:47:48.780421 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:47:48.780570 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.776288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:35825 (local address 127.7.171.1:43169)
0809 04:47:48.776615 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:48.776627 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:48.776715 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:48.776964 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:48.776966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:48.776982 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:48.777066 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.777070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.777976 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.777979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.779200 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.779213 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.780044 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.780047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.780192 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.780220 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 04:47:48.780304 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:47:48.780387 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":34,"thread_start_us":122,"threads_started":1}
I20260809 04:47:48.780689 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.775864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:59675 (local address 127.7.171.3:33131)
0809 04:47:48.776013 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:48.776017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:48.776247 (+ 230us) server_negotiation.cc:407] Connection header received
0809 04:47:48.776398 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:48.776404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:48.776467 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:48.776557 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:48.776899 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.777701 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.779376 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.779943 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.780045 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.780140 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:48.780455 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 04:47:48.780530 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:47:48.780896 23958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.775791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:33131 (local address 127.7.171.1:59675)
0809 04:47:48.776137 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:48.776152 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:48.776274 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:48.776600 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:48.776603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:48.776626 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:48.776782 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.776792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.777813 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.777815 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.779206 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:48.779213 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.780211 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.780216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:48.780337 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.780362 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:47:48.780437 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 04:47:48.780508 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":41,"thread_start_us":141,"threads_started":1}
I20260809 04:47:48.781311 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f747fa95d46648a68b40783ea7c0e537" candidate_uuid: "9ef5af632ba244d98bbea049cb2cdf31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98bcb0a3fa254657b2179f217677d759" is_pre_election: true
I20260809 04:47:48.781461 23875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f747fa95d46648a68b40783ea7c0e537" candidate_uuid: "9ef5af632ba244d98bbea049cb2cdf31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4acd726da84c088082e33d3af47c87" is_pre_election: true
I20260809 04:47:48.781672 23732 raft_consensus.cc:2454] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ef5af632ba244d98bbea049cb2cdf31 in term 0.
I20260809 04:47:48.781697 23875 raft_consensus.cc:2454] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ef5af632ba244d98bbea049cb2cdf31 in term 0.
I20260809 04:47:48.782162 23513 leader_election.cc:302] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a4acd726da84c088082e33d3af47c87, 9ef5af632ba244d98bbea049cb2cdf31; no voters:
I20260809 04:47:48.782431 23957 raft_consensus.cc:2781] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:47:48.782522 23957 raft_consensus.cc:482] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:47:48.782567 23957 raft_consensus.cc:3037] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:48.783705 23957 raft_consensus.cc:504] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } }
I20260809 04:47:48.784104 23957 leader_election.cc:288] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 1 election: Requested vote from peers 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131), 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825)
I20260809 04:47:48.784475 23875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f747fa95d46648a68b40783ea7c0e537" candidate_uuid: "9ef5af632ba244d98bbea049cb2cdf31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4acd726da84c088082e33d3af47c87"
I20260809 04:47:48.784545 23732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f747fa95d46648a68b40783ea7c0e537" candidate_uuid: "9ef5af632ba244d98bbea049cb2cdf31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98bcb0a3fa254657b2179f217677d759"
I20260809 04:47:48.784654 23875 raft_consensus.cc:3037] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:48.784713 23732 raft_consensus.cc:3037] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:48.785768 23875 raft_consensus.cc:2454] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef5af632ba244d98bbea049cb2cdf31 in term 1.
I20260809 04:47:48.785852 23732 raft_consensus.cc:2454] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef5af632ba244d98bbea049cb2cdf31 in term 1.
I20260809 04:47:48.786060 23513 leader_election.cc:302] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a4acd726da84c088082e33d3af47c87, 9ef5af632ba244d98bbea049cb2cdf31; no voters:
I20260809 04:47:48.786249 23957 raft_consensus.cc:2781] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:47:48.786510 23957 raft_consensus.cc:686] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 LEADER]: Becoming Leader. State: Replica: 9ef5af632ba244d98bbea049cb2cdf31, State: Running, Role: LEADER
I20260809 04:47:48.786880 23957 consensus_queue.cc:237] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } }
I20260809 04:47:48.790278 23430 catalog_manager.cc:5495] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ef5af632ba244d98bbea049cb2cdf31 (127.7.171.1). New cstate: current_term: 1 leader_uuid: "9ef5af632ba244d98bbea049cb2cdf31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } health_report { overall_health: UNKNOWN } } }
I20260809 04:47:48.804416 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.800458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53778 (local address 127.7.171.1:37385)
0809 04:47:48.800607 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:48.800612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:48.800645 (+ 33us) server_negotiation.cc:407] Connection header received
0809 04:47:48.800773 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:48.800775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:48.800832 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:48.800942 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:47:48.801222 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.801843 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.802859 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:48.803082 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:48.803197 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:48.803510 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:47:48.803976 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:47:48.804159 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:48.804198 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 04:47:48.804261 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:47:48.810837 23732 raft_consensus.cc:1261] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759 [term 1 FOLLOWER]: Refusing update from remote peer 9ef5af632ba244d98bbea049cb2cdf31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:48.810837 23875 raft_consensus.cc:1261] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87 [term 1 FOLLOWER]: Refusing update from remote peer 9ef5af632ba244d98bbea049cb2cdf31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:48.811655 23957 consensus_queue.cc:1035] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:48.811903 23960 consensus_queue.cc:1035] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:48.829742 23426 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:48.829954 23426 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:48.832654 23426 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Sending DeleteTablet for 3 replicas of tablet f747fa95d46648a68b40783ea7c0e537
I20260809 04:47:48.833537 23563 tablet_service.cc:1514] Processing DeleteTablet for tablet f747fa95d46648a68b40783ea7c0e537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:48 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:53766
I20260809 04:47:48.833604 23854 tablet_service.cc:1514] Processing DeleteTablet for tablet f747fa95d46648a68b40783ea7c0e537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:48 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:59836
I20260809 04:47:48.833840 23709 tablet_service.cc:1514] Processing DeleteTablet for tablet f747fa95d46648a68b40783ea7c0e537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:48 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:37008
I20260809 04:47:48.834160 23976 tablet_replica.cc:330] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87: stopping tablet replica
I20260809 04:47:48.834260 23975 tablet_replica.cc:330] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: stopping tablet replica
I20260809 04:47:48.834506 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 23496
I20260809 04:47:48.834509 23977 tablet_replica.cc:330] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759: stopping tablet replica
I20260809 04:47:48.836267 23977 raft_consensus.cc:2229] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:48.836732 23977 raft_consensus.cc:2258] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:48.838982 23976 raft_consensus.cc:2229] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:48.839099 23977 ts_tablet_manager.cc:1905] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:48.839351 23976 raft_consensus.cc:2258] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:48.840915 23976 ts_tablet_manager.cc:1905] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:48.842046 23977 ts_tablet_manager.cc:1918] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:48.842170 23977 log.cc:1196] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/f747fa95d46648a68b40783ea7c0e537
I20260809 04:47:48.842553 23977 ts_tablet_manager.cc:1939] T f747fa95d46648a68b40783ea7c0e537 P 98bcb0a3fa254657b2179f217677d759: Deleting consensus metadata
I20260809 04:47:48.843675 23976 ts_tablet_manager.cc:1918] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:48.843809 23976 log.cc:1196] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/f747fa95d46648a68b40783ea7c0e537
I20260809 04:47:48.844020 23416 catalog_manager.cc:4841] TS 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825): tablet f747fa95d46648a68b40783ea7c0e537 (table test-table [id=1cde2e9c7b154232a600d1660a0b1a1e]) successfully deleted
I20260809 04:47:48.844199 23976 ts_tablet_manager.cc:1939] T f747fa95d46648a68b40783ea7c0e537 P 2a4acd726da84c088082e33d3af47c87: Deleting consensus metadata
I20260809 04:47:48.845371 23416 catalog_manager.cc:4841] TS 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131): tablet f747fa95d46648a68b40783ea7c0e537 (table test-table [id=1cde2e9c7b154232a600d1660a0b1a1e]) successfully deleted
W20260809 04:47:48.845628 23416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:37385 (error 108)
I20260809 04:47:48.845796 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 23640
I20260809 04:47:48.853459 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:48.853150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:37385 (local address 127.0.0.1:53784)
0809 04:47:48.853298 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:48.853372 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 04:47:48.853708 23416 catalog_manager.cc:4584] TS 9ef5af632ba244d98bbea049cb2cdf31 (127.7.171.1:37385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f747fa95d46648a68b40783ea7c0e537: Network error: Client connection negotiation failed: client connection to 127.7.171.1:37385: connect: Connection refused (error 111)
I20260809 04:47:48.854241 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 23786
I20260809 04:47:48.863184 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 23398
I20260809 04:47:48.872395 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34679
--webserver_interface=127.7.171.62
--webserver_port=40161
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:34679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:48.986711 23979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:48.987017 23979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:48.987109 23979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:48.991283 23979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:48.991375 23979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:48.991416 23979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:48.991474 23979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:48.991511 23979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:48.997088 23979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:34679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=40161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:48.998190 23979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:48.999254 23979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:49.013398 23985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:49.013466 23987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:49.013635 23979 server_base.cc:1034] running on GCE node
W20260809 04:47:49.013502 23989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:49.014379 23979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:49.015578 23979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:49.016744 23979 hybrid_clock.cc:648] HybridClock initialized: now 1786250869016707 us; error 50 us; skew 500 ppm
Aug 09 04:47:49 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250869, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:49.019738 23979 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:47:49.020689 23979 webserver.cc:460] Webserver started at http://127.7.171.62:40161/ using document root <none> and password file <none>
I20260809 04:47:49.021251 23979 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:49.021323 23979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:49.025388 23979 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:47:49.027463 23996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:49.028729 23979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:47:49.028863 23979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "8cfb760004634a788d6176f118079317"
format_stamp: "Formatted at 2026-08-09 04:47:47 on dist-test-slave-1zqn"
server_key: "4c9c43a7dbd891d424f563e552c5a607"
server_key_iv: "109b1ea157d5f89d59c6b7afd244155b"
server_key_version: "encryptionkey@0"
I20260809 04:47:49.029488 23979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:49.047855 23979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:49.051357 23979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:49.051579 23979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:49.059430 23979 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:34679
I20260809 04:47:49.059787 24056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:34679 every 8 connection(s)
I20260809 04:47:49.061003 23979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:47:49.065614 24057 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:47:49.067271 24057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Bootstrap starting.
I20260809 04:47:49.069703 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 23979
I20260809 04:47:49.070228 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:37385
--local_ip_for_outbound_sockets=127.7.171.1
--tserver_master_addrs=127.7.171.62:34679
--webserver_port=34103
--webserver_interface=127.7.171.1
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:47:49.086738 24057 log.cc:824] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:49.095927 24057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:47:49.096329 24057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Bootstrap complete.
I20260809 04:47:49.100732 24057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:49.101308 24057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cfb760004634a788d6176f118079317, State: Initialized, Role: FOLLOWER
I20260809 04:47:49.101811 24057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:49.101962 24057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:47:49.102058 24057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:47:49.102175 24057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:47:49.104197 24057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:49.104533 24057 leader_election.cc:302] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cfb760004634a788d6176f118079317; no voters:
I20260809 04:47:49.104878 24057 leader_election.cc:288] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:47:49.104964 24062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:47:49.105166 24062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 2 LEADER]: Becoming Leader. State: Replica: 8cfb760004634a788d6176f118079317, State: Running, Role: LEADER
I20260809 04:47:49.105563 24062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:49.106240 24057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:49.106527 24064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cfb760004634a788d6176f118079317. Latest consensus state: current_term: 2 leader_uuid: "8cfb760004634a788d6176f118079317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } } }
I20260809 04:47:49.106652 24064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:49.106931 24063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8cfb760004634a788d6176f118079317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } } }
I20260809 04:47:49.107039 24066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:47:49.107040 24063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:49.111903 24066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:47:49.112578 24066 catalog_manager.cc:1259] Loaded cluster ID: f7da400cef5746219880940dd9cf427a
I20260809 04:47:49.112655 24066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:47:49.113736 24066 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 04:47:49.116645 24066 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 04:47:49.121045 24066 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:47:49.121666 24066 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Loaded TSK: 0
I20260809 04:47:49.122597 24066 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:47:49.199039 24060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:49.199313 24060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:49.199359 24060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:49.203502 24060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:49.203568 24060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:49.203627 24060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:47:49.210117 24060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:37385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=34103
--tserver_master_addrs=127.7.171.62:34679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:49.211207 24060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:49.212328 24060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:49.222672 24089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:49.222656 24087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:49.222625 24086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:49.223189 24060 server_base.cc:1034] running on GCE node
I20260809 04:47:49.223690 24060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:49.224501 24060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:49.225704 24060 hybrid_clock.cc:648] HybridClock initialized: now 1786250869225676 us; error 36 us; skew 500 ppm
Aug 09 04:47:49 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250869, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:49.229175 24060 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:47:49.230207 24060 webserver.cc:460] Webserver started at http://127.7.171.1:34103/ using document root <none> and password file <none>
I20260809 04:47:49.230792 24060 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:49.230912 24060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:49.235272 24060 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:47:49.237788 24097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:49.239162 24060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 04:47:49.239307 24060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "9ef5af632ba244d98bbea049cb2cdf31"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "bb8ce3a2c2de02247a13ff72502a9278"
server_key_iv: "c3741a5835ecd2ef58dfcca74235ee8c"
server_key_version: "encryptionkey@0"
I20260809 04:47:49.239895 24060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:49.264381 24060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:49.267907 24060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:49.268182 24060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:49.268869 24060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:49.270005 24104 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 04:47:49.274300 24060 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 04:47:49.274374 24060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260809 04:47:49.274461 24060 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 04:47:49.276237 24060 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 04:47:49.276283 24060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260809 04:47:49.276558 24104 tablet_bootstrap.cc:492] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Bootstrap starting.
I20260809 04:47:49.291843 24060 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:37385
I20260809 04:47:49.291848 24221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:37385 every 8 connection(s)
I20260809 04:47:49.293264 24060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:47:49.298548 24104 log.cc:824] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:49.298668 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 24060
I20260809 04:47:49.299197 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:35825
--local_ip_for_outbound_sockets=127.7.171.2
--tserver_master_addrs=127.7.171.62:34679
--webserver_port=38863
--webserver_interface=127.7.171.2
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:47:49 dist-test-slave-1zqn krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250869, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:49.308401 24104 tablet_bootstrap.cc:492] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:47:49.308910 24104 tablet_bootstrap.cc:492] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Bootstrap complete.
I20260809 04:47:49.310608 24104 ts_tablet_manager.cc:1397] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Time spent bootstrapping tablet: real 0.034s user 0.017s sys 0.013s
I20260809 04:47:49.313565 24104 raft_consensus.cc:348] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } }
I20260809 04:47:49.314095 24104 raft_consensus.cc:729] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef5af632ba244d98bbea049cb2cdf31, State: Initialized, Role: FOLLOWER
I20260809 04:47:49.314462 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.296270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:37397)
0809 04:47:49.296852 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:49.296899 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:49.297758 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:49.299213 (+ 1455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:49.299224 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:49.299822 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:49.300560 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.300595 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.302262 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.302266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.303294 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.303306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.304401 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.304408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.304643 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.305384 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:49.305408 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:49.307600 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:49.310550 (+ 2950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:49.310560 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:49.310577 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:49.311012 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:49.311457 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:49.311460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:49.311462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:49.311665 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:49.312469 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:49.312478 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:49.312823 (+ 345us) client_negotiation.cc:769] Sending connection context
0809 04:47:49.313059 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 04:47:49.313474 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":172,"threads_started":1}
I20260809 04:47:49.314630 24104 consensus_queue.cc:260] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } } peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } }
I20260809 04:47:49.314891 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.296171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:37397 (local address 127.7.171.62:34679)
0809 04:47:49.296547 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:49.296563 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:49.297027 (+ 464us) server_negotiation.cc:407] Connection header received
0809 04:47:49.298203 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:49.298215 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:49.298540 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:49.298906 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:49.300797 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.302090 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.303437 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.304220 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.304382 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.307858 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:49.307866 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:49.307881 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:49.307917 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:49.310404 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:49.311159 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:49.311166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:49.311172 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:49.311338 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:49.311785 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:49.311789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:49.311790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:49.312332 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:49.312475 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:49.313204 (+ 729us) server_negotiation.cc:299] Negotiation successful
0809 04:47:49.313518 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":127,"threads_started":1}
I20260809 04:47:49.315658 24104 ts_tablet_manager.cc:1428] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:47:49.317941 24223 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34679
I20260809 04:47:49.318194 24223 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:49.318722 24223 heartbeater.cc:511] Master 127.7.171.62:34679 requested a full tablet report, sending...
I20260809 04:47:49.321339 24015 ts_manager.cc:194] Registered new tserver with Master: 9ef5af632ba244d98bbea049cb2cdf31 (127.7.171.1:37385)
I20260809 04:47:49.325371 24015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:37397
I20260809 04:47:49.327916 24223 heartbeater.cc:503] Master 127.7.171.62:34679 was elected leader, sending a full tablet report...
Aug 09 04:47:49 dist-test-slave-1zqn krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250869, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:47:49.335840 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.324684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:37385 (local address 127.0.0.1:53788)
0809 04:47:49.325527 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:49.325576 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:49.325749 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:49.326348 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:49.326355 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:49.326827 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:49.326960 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.326969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.328344 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.328348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.329250 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.329261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.329972 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.329976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.330055 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.330616 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:49.330629 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:49.332049 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:49.334223 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:49.334230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:49.334234 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:49.334460 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:49.334770 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:49.334773 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:49.334774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:49.334850 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:49.335351 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:49.335355 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:49.335454 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 04:47:49.335563 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 04:47:49.335714 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":111,"threads_started":1}
I20260809 04:47:49.336108 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.324907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53788 (local address 127.7.171.1:37385)
0809 04:47:49.325271 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:49.325283 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:49.325684 (+ 401us) server_negotiation.cc:407] Connection header received
0809 04:47:49.325915 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:49.325921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:49.326075 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:49.326230 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:49.327103 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.328205 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.329378 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.329852 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.329964 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.332271 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:49.332281 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:49.332293 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:49.332318 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:49.334117 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:49.334576 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:49.334581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:49.334584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:49.334677 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:49.334936 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:49.334939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:49.334940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:49.335242 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:49.335370 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:49.335663 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 04:47:49.335915 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":95,"threads_started":1}
I20260809 04:47:49.336997 24130 tablet_service.cc:1514] Processing DeleteTablet for tablet f747fa95d46648a68b40783ea7c0e537 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:48 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:53788
I20260809 04:47:49.338011 24235 tablet_replica.cc:330] stopping tablet replica
I20260809 04:47:49.338328 24235 raft_consensus.cc:2229] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:49.338580 24235 raft_consensus.cc:2258] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:49.339560 24235 ts_tablet_manager.cc:1905] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:49.343178 24235 ts_tablet_manager.cc:1918] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:49.343262 24235 log.cc:1196] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal/wals/f747fa95d46648a68b40783ea7c0e537
I20260809 04:47:49.343577 24235 ts_tablet_manager.cc:1939] T f747fa95d46648a68b40783ea7c0e537 P 9ef5af632ba244d98bbea049cb2cdf31: Deleting consensus metadata
I20260809 04:47:49.345299 23998 catalog_manager.cc:4841] TS 9ef5af632ba244d98bbea049cb2cdf31 (127.7.171.1:37385): tablet f747fa95d46648a68b40783ea7c0e537 (table test-table [id=1cde2e9c7b154232a600d1660a0b1a1e]) successfully deleted
W20260809 04:47:49.424356 24229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:49.424643 24229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:49.424718 24229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:49.428812 24229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:49.428893 24229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:49.428987 24229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:47:49.434275 24229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:35825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=38863
--tserver_master_addrs=127.7.171.62:34679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:49.435335 24229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:49.436384 24229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:49.446489 24241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:49.446508 24240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:49.446554 24243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:49.446811 24229 server_base.cc:1034] running on GCE node
I20260809 04:47:49.447329 24229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:49.448030 24229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:49.449280 24229 hybrid_clock.cc:648] HybridClock initialized: now 1786250869449257 us; error 47 us; skew 500 ppm
Aug 09 04:47:49 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250869, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:49.452245 24229 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:47:49.453073 24229 webserver.cc:460] Webserver started at http://127.7.171.2:38863/ using document root <none> and password file <none>
I20260809 04:47:49.453590 24229 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:49.453672 24229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:49.457505 24229 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 04:47:49.459610 24251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:49.460908 24229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:47:49.461006 24229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "98bcb0a3fa254657b2179f217677d759"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "81c244db71210b1446ed99d39716310f"
server_key_iv: "0cf8c9eb61aa10df473214b6d9d179fb"
server_key_version: "encryptionkey@0"
I20260809 04:47:49.461530 24229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:49.483799 24229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:49.487339 24229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:49.487535 24229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:49.488210 24229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:49.489084 24229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:49.489152 24229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:49.489209 24229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:49.489233 24229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:49.498821 24229 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:35825
I20260809 04:47:49.498850 24376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:35825 every 8 connection(s)
I20260809 04:47:49.499917 24229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:47:49.508893 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 24229
I20260809 04:47:49.509353 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:33131
--local_ip_for_outbound_sockets=127.7.171.3
--tserver_master_addrs=127.7.171.62:34679
--webserver_port=39051
--webserver_interface=127.7.171.3
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:47:49 dist-test-slave-1zqn krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250869, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:49.514124 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.501790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:58707 (local address 127.7.171.62:34679)
0809 04:47:49.501957 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:49.501961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:49.502867 (+ 906us) server_negotiation.cc:407] Connection header received
0809 04:47:49.504152 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:49.504155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:49.504209 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:49.504285 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:49.505429 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.505972 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.507229 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.507462 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.507542 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.510246 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:49.510253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:49.510259 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:49.510294 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:49.511855 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:49.512524 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:49.512527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:49.512529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:49.512623 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:49.513014 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:49.513016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:49.513018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:49.513196 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:49.513277 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:49.513880 (+ 603us) server_negotiation.cc:299] Negotiation successful
0809 04:47:49.513996 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:47:49.515388 24380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.502262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:58707)
0809 04:47:49.502708 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:49.502752 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:49.503912 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:49.504451 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:49.504457 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:49.504806 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:49.505289 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.505302 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.506089 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.506092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.507073 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.507089 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.507595 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.507598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.507800 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.508451 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:49.508467 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:49.510070 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:49.511978 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:49.511984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:49.512001 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:49.512408 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:49.512723 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:49.512729 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:49.512734 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:49.512918 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:49.513305 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:49.513315 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:49.513629 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 04:47:49.513844 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 04:47:49.514238 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":121,"threads_started":1}
I20260809 04:47:49.516786 24377 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34679
I20260809 04:47:49.517050 24377 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:49.517547 24377 heartbeater.cc:511] Master 127.7.171.62:34679 requested a full tablet report, sending...
I20260809 04:47:49.518690 24015 ts_manager.cc:194] Registered new tserver with Master: 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825)
I20260809 04:47:49.519270 24015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:58707
W20260809 04:47:49.626299 24381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:49.626616 24381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:49.626700 24381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:49.630764 24381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:49.630853 24381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:49.630944 24381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:47:49.636322 24381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:33131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=39051
--tserver_master_addrs=127.7.171.62:34679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:49.637382 24381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:49.638422 24381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:49.648439 24388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:49.648510 24381 server_base.cc:1034] running on GCE node
W20260809 04:47:49.648412 24390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:49.648422 24387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:49.649128 24381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:49.649791 24381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:49.650988 24381 hybrid_clock.cc:648] HybridClock initialized: now 1786250869650973 us; error 32 us; skew 500 ppm
Aug 09 04:47:49 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250869, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:49.653801 24381 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:47:49.654661 24381 webserver.cc:460] Webserver started at http://127.7.171.3:39051/ using document root <none> and password file <none>
I20260809 04:47:49.655251 24381 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:49.655366 24381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:49.659260 24381 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:47:49.661343 24397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:49.662573 24381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:47:49.662714 24381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "2a4acd726da84c088082e33d3af47c87"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "3fff5f4ba32a4bfdc06e0c6a6aec2a37"
server_key_iv: "7dd2c4eb21cf2dd51d41f1e630a21a51"
server_key_version: "encryptionkey@0"
I20260809 04:47:49.663270 24381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:49.680812 24381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:49.683787 24381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:49.684003 24381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:49.684646 24381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:49.685498 24381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:49.685572 24381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:49.685640 24381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:49.685693 24381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:49.694923 24381 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:33131
I20260809 04:47:49.694953 24520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:33131 every 8 connection(s)
I20260809 04:47:49.695986 24381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:47:49.697330 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 24381
I20260809 04:47:49.707235 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.698163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39224 (local address 127.7.171.62:34679)
0809 04:47:49.698382 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:49.698386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:49.698569 (+ 183us) server_negotiation.cc:407] Connection header received
0809 04:47:49.698735 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:49.698737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:49.698781 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:49.698860 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:49.699858 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.700769 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.701638 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.701958 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.702067 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.703707 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:49.703711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:49.703715 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:49.703742 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:49.705484 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:49.705919 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:49.705925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:49.705929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:49.706032 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:49.706367 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:49.706371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:49.706372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:49.706548 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:49.706777 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:49.706905 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 04:47:49.707031 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":78,"threads_started":1}
Aug 09 04:47:49 dist-test-slave-1zqn krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250869, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:47:49.711815 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.698075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:56437 (local address 127.7.171.62:34679)
0809 04:47:49.698241 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:49.698246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:49.699310 (+ 1064us) server_negotiation.cc:407] Connection header received
0809 04:47:49.700985 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:49.700988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:49.701032 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:49.701114 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:49.702579 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.703392 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.705165 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.705373 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.705455 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.708262 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:49.708266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:49.708268 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:49.708292 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:49.709815 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:49.710328 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:49.710331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:49.710333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:49.710416 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:49.710770 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:49.710773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:49.710775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:49.710944 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:49.711049 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:49.711550 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 04:47:49.711665 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:47:49.712791 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.698599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:56437)
0809 04:47:49.699171 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:49.699212 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:49.700782 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:49.701308 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:49.701315 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:49.701743 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:49.702403 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.702425 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.703528 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.703531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.705021 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.705034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.705535 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.705543 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.705742 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.706378 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:49.706400 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:49.708037 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:49.709929 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:49.709935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:49.709949 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:49.710215 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:49.710509 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:49.710512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:49.710514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:49.710666 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:49.711050 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:49.711058 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:49.711306 (+ 248us) client_negotiation.cc:769] Sending connection context
0809 04:47:49.711492 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 04:47:49.711833 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":172,"threads_started":1}
I20260809 04:47:49.714578 24521 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34679
I20260809 04:47:49.714828 24521 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:49.715327 24521 heartbeater.cc:511] Master 127.7.171.62:34679 requested a full tablet report, sending...
I20260809 04:47:49.716374 24015 ts_manager.cc:194] Registered new tserver with Master: 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131)
I20260809 04:47:49.716939 24015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:56437
I20260809 04:47:49.721287 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:47:49.725438 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.721960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39240 (local address 127.7.171.62:34679)
0809 04:47:49.722074 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:49.722077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:49.722280 (+ 203us) server_negotiation.cc:407] Connection header received
0809 04:47:49.722406 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:49.722408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:49.722498 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:49.722582 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:47:49.722783 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.723258 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.724102 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.724325 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.724403 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.724706 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:47:49.725055 (+ 349us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:47:49.725148 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:49.725232 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 04:47:49.725300 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 04:47:49.726086 24015 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:47:49.729301 24015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:47:49.737354 24130 tablet_service.cc:1467] Processing CreateTablet for tablet 5136603a0ca449ba9c720e0ab8be229f (DEFAULT_TABLE table=test-table [id=0ec1779b53a4460680fe17ea2ae744b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:49.738054 24130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5136603a0ca449ba9c720e0ab8be229f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:49.741276 24104 tablet_bootstrap.cc:492] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: Bootstrap starting.
I20260809 04:47:49.742158 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.736018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:33131 (local address 127.0.0.1:59844)
0809 04:47:49.736143 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:49.736160 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:49.736242 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:49.737572 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:49.737575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:49.737610 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:49.737764 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.737773 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.739262 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.739265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.740483 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.740494 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.741631 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.741635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.741740 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.741830 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 04:47:49.741896 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 04:47:49.741992 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":71}
I20260809 04:47:49.742260 24104 tablet_bootstrap.cc:654] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:49.742478 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.736558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37024 (local address 127.7.171.2:35825)
0809 04:47:49.737042 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:49.737056 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:49.737080 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:47:49.737168 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:49.737171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:49.737395 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:49.737572 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:49.737991 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.739068 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.740420 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.741371 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.741480 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.741610 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:49.741966 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 04:47:49.742071 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":169,"threads_started":1}
I20260809 04:47:49.742563 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.736386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:35825 (local address 127.0.0.1:37024)
0809 04:47:49.736652 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:49.736669 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:49.736765 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:49.737568 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:49.737572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:49.737592 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:49.737870 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.737874 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.739227 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.739232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.740307 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.740319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.741672 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.741675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.741741 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.741819 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 04:47:49.741913 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 04:47:49.742021 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":146,"thread_start_us":107,"threads_started":1}
I20260809 04:47:49.742595 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.736285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59844 (local address 127.7.171.3:33131)
0809 04:47:49.737042 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:49.737056 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:49.737080 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:47:49.737165 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:49.737171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:49.737395 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:49.738098 (+ 703us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:49.738274 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.739124 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.740634 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.741486 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.742096 (+ 610us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.742262 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:49.742319 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 04:47:49.742428 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":195,"threads_started":1}
I20260809 04:47:49.743973 24104 tablet_bootstrap.cc:492] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: No bootstrap required, opened a new log
I20260809 04:47:49.744081 24104 ts_tablet_manager.cc:1397] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260809 04:47:49.744526 24104 raft_consensus.cc:348] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:49.744658 24104 raft_consensus.cc:374] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:49.744695 24104 raft_consensus.cc:729] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef5af632ba244d98bbea049cb2cdf31, State: Initialized, Role: FOLLOWER
I20260809 04:47:49.744850 24104 consensus_queue.cc:260] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:49.745201 24104 ts_tablet_manager.cc:1428] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:47:49.745868 24452 tablet_service.cc:1467] Processing CreateTablet for tablet 5136603a0ca449ba9c720e0ab8be229f (DEFAULT_TABLE table=test-table [id=0ec1779b53a4460680fe17ea2ae744b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:49.745869 24305 tablet_service.cc:1467] Processing CreateTablet for tablet 5136603a0ca449ba9c720e0ab8be229f (DEFAULT_TABLE table=test-table [id=0ec1779b53a4460680fe17ea2ae744b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:49.746747 24452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5136603a0ca449ba9c720e0ab8be229f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:49.746773 24305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5136603a0ca449ba9c720e0ab8be229f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:49.753858 24537 tablet_bootstrap.cc:492] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87: Bootstrap starting.
I20260809 04:47:49.754810 24538 tablet_bootstrap.cc:492] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759: Bootstrap starting.
I20260809 04:47:49.756479 24537 tablet_bootstrap.cc:654] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:49.756949 24538 tablet_bootstrap.cc:654] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:49.757274 24537 log.cc:824] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:49.757673 24538 log.cc:824] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:49.759116 24537 tablet_bootstrap.cc:492] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87: No bootstrap required, opened a new log
I20260809 04:47:49.759325 24537 ts_tablet_manager.cc:1397] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:47:49.759487 24538 tablet_bootstrap.cc:492] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759: No bootstrap required, opened a new log
I20260809 04:47:49.759685 24538 ts_tablet_manager.cc:1397] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:47:49.762425 24537 raft_consensus.cc:348] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:49.762697 24537 raft_consensus.cc:374] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:49.762768 24537 raft_consensus.cc:729] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4acd726da84c088082e33d3af47c87, State: Initialized, Role: FOLLOWER
I20260809 04:47:49.762619 24538 raft_consensus.cc:348] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:49.762923 24538 raft_consensus.cc:374] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:49.763002 24538 raft_consensus.cc:729] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98bcb0a3fa254657b2179f217677d759, State: Initialized, Role: FOLLOWER
I20260809 04:47:49.763249 24537 consensus_queue.cc:260] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:49.763362 24538 consensus_queue.cc:260] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:49.763916 24521 heartbeater.cc:503] Master 127.7.171.62:34679 was elected leader, sending a full tablet report...
I20260809 04:47:49.764015 24377 heartbeater.cc:503] Master 127.7.171.62:34679 was elected leader, sending a full tablet report...
I20260809 04:47:49.764364 24537 ts_tablet_manager.cc:1428] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 04:47:49.764379 24538 ts_tablet_manager.cc:1428] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260809 04:47:49.794684 24224 tablet.cc:2369] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:47:49.798220 24231 raft_consensus.cc:482] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:49.798441 24231 raft_consensus.cc:504] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:49.799659 24231 leader_election.cc:288] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131), 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825)
I20260809 04:47:49.804484 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.800282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:55689 (local address 127.7.171.2:35825)
0809 04:47:49.800417 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:49.800423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:49.800535 (+ 112us) server_negotiation.cc:407] Connection header received
0809 04:47:49.800618 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:49.800633 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:49.800697 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:49.800782 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:49.801021 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.801721 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.803131 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.803751 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.803863 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.803929 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:49.804296 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 04:47:49.804377 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:47:49.804556 24227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.799857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:33131 (local address 127.7.171.1:40087)
0809 04:47:49.799993 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:49.800011 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:49.800148 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:49.800442 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:49.800446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:49.800471 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:49.800626 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.800632 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.801647 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.801649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.802993 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.803005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.803875 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.803881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.804081 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.804174 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 04:47:49.804258 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:47:49.804393 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":134}
I20260809 04:47:49.804579 24543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.800163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:35825 (local address 127.7.171.1:55689)
0809 04:47:49.800442 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:49.800454 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:49.800535 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:49.800799 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:49.800801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:49.800818 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:49.800914 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.800918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.801810 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.801813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.802993 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:49.803005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.803895 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.803898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:49.804041 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.804174 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 04:47:49.804258 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:47:49.804393 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":138,"thread_start_us":101,"threads_started":1}
I20260809 04:47:49.804899 24531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.800005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:40087 (local address 127.7.171.3:33131)
0809 04:47:49.800153 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:49.800158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:49.800174 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:47:49.800282 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:49.800284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:49.800342 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:49.800435 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:49.800744 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.801535 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.803137 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.803750 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.803863 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.803929 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:49.804698 (+ 769us) server_negotiation.cc:299] Negotiation successful
0809 04:47:49.804791 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:47:49.805460 24328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5136603a0ca449ba9c720e0ab8be229f" candidate_uuid: "9ef5af632ba244d98bbea049cb2cdf31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98bcb0a3fa254657b2179f217677d759" is_pre_election: true
I20260809 04:47:49.805487 24472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5136603a0ca449ba9c720e0ab8be229f" candidate_uuid: "9ef5af632ba244d98bbea049cb2cdf31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4acd726da84c088082e33d3af47c87" is_pre_election: true
I20260809 04:47:49.805761 24328 raft_consensus.cc:2454] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ef5af632ba244d98bbea049cb2cdf31 in term 0.
I20260809 04:47:49.805787 24472 raft_consensus.cc:2454] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ef5af632ba244d98bbea049cb2cdf31 in term 0.
I20260809 04:47:49.806282 24099 leader_election.cc:302] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98bcb0a3fa254657b2179f217677d759, 9ef5af632ba244d98bbea049cb2cdf31; no voters:
I20260809 04:47:49.806571 24231 raft_consensus.cc:2781] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:47:49.806671 24231 raft_consensus.cc:482] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:47:49.806720 24231 raft_consensus.cc:3037] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:49.807905 24231 raft_consensus.cc:504] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:49.808367 24231 leader_election.cc:288] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 1 election: Requested vote from peers 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131), 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825)
I20260809 04:47:49.808761 24472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5136603a0ca449ba9c720e0ab8be229f" candidate_uuid: "9ef5af632ba244d98bbea049cb2cdf31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a4acd726da84c088082e33d3af47c87"
I20260809 04:47:49.808830 24328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5136603a0ca449ba9c720e0ab8be229f" candidate_uuid: "9ef5af632ba244d98bbea049cb2cdf31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98bcb0a3fa254657b2179f217677d759"
I20260809 04:47:49.808899 24472 raft_consensus.cc:3037] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:49.808938 24328 raft_consensus.cc:3037] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:49.810213 24472 raft_consensus.cc:2454] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef5af632ba244d98bbea049cb2cdf31 in term 1.
I20260809 04:47:49.810212 24328 raft_consensus.cc:2454] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ef5af632ba244d98bbea049cb2cdf31 in term 1.
I20260809 04:47:49.810491 24099 leader_election.cc:302] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a4acd726da84c088082e33d3af47c87, 9ef5af632ba244d98bbea049cb2cdf31; no voters:
I20260809 04:47:49.810655 24231 raft_consensus.cc:2781] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:47:49.810883 24231 raft_consensus.cc:686] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 LEADER]: Becoming Leader. State: Replica: 9ef5af632ba244d98bbea049cb2cdf31, State: Running, Role: LEADER
I20260809 04:47:49.811165 24231 consensus_queue.cc:237] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:49.814468 24015 catalog_manager.cc:5495] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ef5af632ba244d98bbea049cb2cdf31 (127.7.171.1). New cstate: current_term: 1 leader_uuid: "9ef5af632ba244d98bbea049cb2cdf31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } health_report { overall_health: HEALTHY } } }
I20260809 04:47:49.842268 24546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.838456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53800 (local address 127.7.171.1:37385)
0809 04:47:49.838771 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:49.838775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:49.838787 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:47:49.838864 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:49.838866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:49.838920 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:49.839048 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:47:49.839304 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.839823 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.840768 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:49.840999 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:49.841110 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:49.841396 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:47:49.841843 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:47:49.841976 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:49.842037 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 04:47:49.842096 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":93,"threads_started":1}
I20260809 04:47:49.846635 24472 raft_consensus.cc:1261] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87 [term 1 FOLLOWER]: Refusing update from remote peer 9ef5af632ba244d98bbea049cb2cdf31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:49.846635 24328 raft_consensus.cc:1261] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759 [term 1 FOLLOWER]: Refusing update from remote peer 9ef5af632ba244d98bbea049cb2cdf31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:49.847412 24231 consensus_queue.cc:1035] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:49.847594 24544 consensus_queue.cc:1035] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:49.864321 24014 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:49.864527 24014 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:49.867053 24014 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Sending DeleteTablet for 3 replicas of tablet 5136603a0ca449ba9c720e0ab8be229f
I20260809 04:47:49.867867 24452 tablet_service.cc:1514] Processing DeleteTablet for tablet 5136603a0ca449ba9c720e0ab8be229f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:49 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:59844
I20260809 04:47:49.867883 24305 tablet_service.cc:1514] Processing DeleteTablet for tablet 5136603a0ca449ba9c720e0ab8be229f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:49 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:37024
I20260809 04:47:49.868091 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 24060
I20260809 04:47:49.868145 24130 tablet_service.cc:1514] Processing DeleteTablet for tablet 5136603a0ca449ba9c720e0ab8be229f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:49 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:53788
I20260809 04:47:49.869830 24557 tablet_replica.cc:330] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759: stopping tablet replica
I20260809 04:47:49.870191 24557 raft_consensus.cc:2229] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:49.870517 24557 raft_consensus.cc:2258] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:49.870911 24558 tablet_replica.cc:330] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87: stopping tablet replica
I20260809 04:47:49.871299 24558 raft_consensus.cc:2229] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:49.871598 24558 raft_consensus.cc:2258] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:49.873271 24558 ts_tablet_manager.cc:1905] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:49.873755 24557 ts_tablet_manager.cc:1905] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:49.875582 24558 ts_tablet_manager.cc:1918] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:49.875725 24558 log.cc:1196] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/5136603a0ca449ba9c720e0ab8be229f
I20260809 04:47:49.876138 24558 ts_tablet_manager.cc:1939] T 5136603a0ca449ba9c720e0ab8be229f P 2a4acd726da84c088082e33d3af47c87: Deleting consensus metadata
I20260809 04:47:49.877193 24557 ts_tablet_manager.cc:1918] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:49.877322 24557 log.cc:1196] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/5136603a0ca449ba9c720e0ab8be229f
I20260809 04:47:49.877336 23998 catalog_manager.cc:4841] TS 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131): tablet 5136603a0ca449ba9c720e0ab8be229f (table test-table [id=0ec1779b53a4460680fe17ea2ae744b0]) successfully deleted
I20260809 04:47:49.877710 24557 ts_tablet_manager.cc:1939] T 5136603a0ca449ba9c720e0ab8be229f P 98bcb0a3fa254657b2179f217677d759: Deleting consensus metadata
I20260809 04:47:49.878903 23998 catalog_manager.cc:4841] TS 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825): tablet 5136603a0ca449ba9c720e0ab8be229f (table test-table [id=0ec1779b53a4460680fe17ea2ae744b0]) successfully deleted
W20260809 04:47:49.880183 23998 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:37385 (error 108)
I20260809 04:47:49.880275 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 24229
I20260809 04:47:49.889533 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 24381
I20260809 04:47:49.891124 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:49.889723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:37385 (local address 127.0.0.1:53804)
0809 04:47:49.890870 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:49.890986 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1024}
W20260809 04:47:49.892589 23998 catalog_manager.cc:4584] TS 9ef5af632ba244d98bbea049cb2cdf31 (127.7.171.1:37385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5136603a0ca449ba9c720e0ab8be229f: Network error: Client connection negotiation failed: client connection to 127.7.171.1:37385: connect: Connection refused (error 111)
I20260809 04:47:49.898509 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 23979
I20260809 04:47:49.907219 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34679
--webserver_interface=127.7.171.62
--webserver_port=40161
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:34679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:50.022385 24560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:50.022671 24560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:50.022713 24560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:50.027263 24560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:50.027335 24560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:50.027355 24560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:50.027411 24560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:50.027439 24560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:50.032876 24560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:34679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=40161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:50.034070 24560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:50.035145 24560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:50.044126 24568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:50.044162 24566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:50.044138 24565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:50.044400 24560 server_base.cc:1034] running on GCE node
I20260809 04:47:50.045094 24560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:50.046264 24560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:50.047454 24560 hybrid_clock.cc:648] HybridClock initialized: now 1786250870047433 us; error 33 us; skew 500 ppm
Aug 09 04:47:50 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:50.050236 24560 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:47:50.051160 24560 webserver.cc:460] Webserver started at http://127.7.171.62:40161/ using document root <none> and password file <none>
I20260809 04:47:50.051733 24560 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:50.051827 24560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:50.055893 24560 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:47:50.058147 24575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:50.059398 24560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:47:50.059546 24560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "8cfb760004634a788d6176f118079317"
format_stamp: "Formatted at 2026-08-09 04:47:47 on dist-test-slave-1zqn"
server_key: "4c9c43a7dbd891d424f563e552c5a607"
server_key_iv: "109b1ea157d5f89d59c6b7afd244155b"
server_key_version: "encryptionkey@0"
I20260809 04:47:50.060220 24560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:50.074657 24560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:50.077718 24560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:50.077932 24560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:50.085616 24560 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:34679
I20260809 04:47:50.085920 24632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:34679 every 8 connection(s)
I20260809 04:47:50.086848 24560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:47:50.091171 24633 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:47:50.092871 24633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Bootstrap starting.
I20260809 04:47:50.094702 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 24560
I20260809 04:47:50.095148 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:37385
--local_ip_for_outbound_sockets=127.7.171.1
--tserver_master_addrs=127.7.171.62:34679
--webserver_port=34103
--webserver_interface=127.7.171.1
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:47:50.111656 24633 log.cc:824] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:50.123126 24633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:47:50.123524 24633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317: Bootstrap complete.
I20260809 04:47:50.128260 24633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:50.128691 24633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cfb760004634a788d6176f118079317, State: Initialized, Role: FOLLOWER
I20260809 04:47:50.129194 24633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:50.129339 24633 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:47:50.129431 24633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:47:50.129552 24633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 2 FOLLOWER]: Advancing to term 3
I20260809 04:47:50.131451 24633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:50.131788 24633 leader_election.cc:302] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cfb760004634a788d6176f118079317; no voters:
I20260809 04:47:50.132176 24633 leader_election.cc:288] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 04:47:50.132258 24637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 04:47:50.132437 24637 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [term 3 LEADER]: Becoming Leader. State: Replica: 8cfb760004634a788d6176f118079317, State: Running, Role: LEADER
I20260809 04:47:50.132732 24637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } }
I20260809 04:47:50.133422 24633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:50.133692 24640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cfb760004634a788d6176f118079317. Latest consensus state: current_term: 3 leader_uuid: "8cfb760004634a788d6176f118079317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } } }
I20260809 04:47:50.133818 24640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:50.134066 24638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8cfb760004634a788d6176f118079317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfb760004634a788d6176f118079317" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34679 } } }
I20260809 04:47:50.134176 24638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfb760004634a788d6176f118079317 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:50.134202 24642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:47:50.139489 24642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:47:50.140324 24642 catalog_manager.cc:1259] Loaded cluster ID: f7da400cef5746219880940dd9cf427a
I20260809 04:47:50.140393 24642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 04:47:50.141477 24642 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fcd1ada29ad google::LogMessage::Fail() at ??:0
@ 0x7fcd1ada588a google::LogMessage::SendToLog() at ??:0
@ 0x7fcd1ada24de google::LogMessage::Flush() at ??:0
@ 0x7fcd1ada3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fcd256e1fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fcd256e2d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fcd256df579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fcd25723640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fcd1cff43d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fcd1cff4c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fcd1cff63f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fcd1cfe5512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fcd1e6296db start_thread at ??:0
@ 0x7fcd198a061f clone at ??:0
W20260809 04:47:50.232417 24635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:50.232738 24635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:50.232834 24635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:50.237615 24635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:50.237708 24635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:50.237803 24635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:47:50.245329 24635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:37385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=34103
--tserver_master_addrs=127.7.171.62:34679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:50.246735 24635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:50.248020 24635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:50.260217 24681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:50.260171 24679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:50.260425 24678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:50.261665 24635 server_base.cc:1034] running on GCE node
I20260809 04:47:50.262151 24635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:50.262837 24635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:50.264041 24635 hybrid_clock.cc:648] HybridClock initialized: now 1786250870264014 us; error 46 us; skew 500 ppm
Aug 09 04:47:50 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:50.267097 24635 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:47:50.268136 24635 webserver.cc:460] Webserver started at http://127.7.171.1:34103/ using document root <none> and password file <none>
I20260809 04:47:50.268734 24635 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:50.268819 24635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:50.273085 24635 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:47:50.275538 24689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:50.277014 24635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:47:50.277176 24635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "9ef5af632ba244d98bbea049cb2cdf31"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "bb8ce3a2c2de02247a13ff72502a9278"
server_key_iv: "c3741a5835ecd2ef58dfcca74235ee8c"
server_key_version: "encryptionkey@0"
I20260809 04:47:50.277890 24635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:50.298063 24635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:50.301555 24635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:50.301775 24635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:50.302424 24635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:50.303664 24697 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 04:47:50.309320 24635 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 04:47:50.309412 24635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 04:47:50.309499 24635 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 04:47:50.311313 24635 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 04:47:50.311388 24635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 04:47:50.311746 24697 tablet_bootstrap.cc:492] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: Bootstrap starting.
I20260809 04:47:50.322587 24635 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:37385
I20260809 04:47:50.322660 24813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:37385 every 8 connection(s)
I20260809 04:47:50.323681 24635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:47:50.333662 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 24635
I20260809 04:47:50.334116 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:35825
--local_ip_for_outbound_sockets=127.7.171.2
--tserver_master_addrs=127.7.171.62:34679
--webserver_port=38863
--webserver_interface=127.7.171.2
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:47:50.334450 24697 log.cc:824] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:50.344283 24697 tablet_bootstrap.cc:492] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:47:50.344766 24697 tablet_bootstrap.cc:492] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: Bootstrap complete.
I20260809 04:47:50.346410 24697 ts_tablet_manager.cc:1397] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: Time spent bootstrapping tablet: real 0.035s user 0.021s sys 0.011s
I20260809 04:47:50.349890 24697 raft_consensus.cc:348] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:50.350452 24697 raft_consensus.cc:729] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ef5af632ba244d98bbea049cb2cdf31, State: Initialized, Role: FOLLOWER
I20260809 04:47:50.351053 24697 consensus_queue.cc:260] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:50.352252 24697 ts_tablet_manager.cc:1428] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260809 04:47:50.387245 24816 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:47:50.325719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:49135)
0809 04:47:50.326390 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:50.326446 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:50.327267 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:50.386275 (+ 59008us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":151,"threads_started":1}
W20260809 04:47:50.387799 24814 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:34679 (0 consecutive failures): Network error: Failed to ping master at 127.7.171.62:34679: Client connection negotiation failed: client connection to 127.7.171.62:34679: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 04:47:50.388665 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:50.388365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:43585)
0809 04:47:50.388512 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:50.388597 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 04:47:50.389423 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:50.389207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:57707)
0809 04:47:50.389317 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:50.389368 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 04:47:50.389628 24814 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:47:50.455521 24818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:50.455775 24818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:50.455819 24818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:50.459905 24818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:50.459975 24818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:50.460091 24818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:47:50.465464 24818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:35825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=38863
--tserver_master_addrs=127.7.171.62:34679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:50.466528 24818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:50.467605 24818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:47:50.477782 24818 server_base.cc:1034] running on GCE node
W20260809 04:47:50.477730 24826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:50.477710 24824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:50.477748 24828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:50.478483 24818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:50.479120 24818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:50.480309 24818 hybrid_clock.cc:648] HybridClock initialized: now 1786250870480289 us; error 30 us; skew 500 ppm
Aug 09 04:47:50 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:50.483119 24818 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:47:50.484022 24818 webserver.cc:460] Webserver started at http://127.7.171.2:38863/ using document root <none> and password file <none>
I20260809 04:47:50.484650 24818 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:50.484742 24818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:50.488672 24818 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:47:50.490792 24835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:50.492036 24818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:47:50.492233 24818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "98bcb0a3fa254657b2179f217677d759"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "81c244db71210b1446ed99d39716310f"
server_key_iv: "0cf8c9eb61aa10df473214b6d9d179fb"
server_key_version: "encryptionkey@0"
I20260809 04:47:50.492803 24818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:50.521590 24818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:50.524683 24818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:50.524899 24818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:50.525529 24818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:50.526701 24818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:50.526777 24818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:50.526845 24818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:50.526876 24818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:50.536780 24818 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:35825
I20260809 04:47:50.536805 24958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:35825 every 8 connection(s)
I20260809 04:47:50.537846 24818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:47:50.541859 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:50.539970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:49659)
0809 04:47:50.540485 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:50.540607 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":119,"threads_started":1}
W20260809 04:47:50.542372 24959 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:34679 (0 consecutive failures): Network error: Failed to ping master at 127.7.171.62:34679: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
I20260809 04:47:50.542986 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:50.542775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:33401)
0809 04:47:50.542885 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:50.542933 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 04:47:50.543656 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:50.543462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:46295)
0809 04:47:50.543565 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:50.543607 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 04:47:50.543843 24959 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 04:47:50.545377 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 24818
I20260809 04:47:50.545783 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:33131
--local_ip_for_outbound_sockets=127.7.171.3
--tserver_master_addrs=127.7.171.62:34679
--webserver_port=39051
--webserver_interface=127.7.171.3
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:45433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:50.659211 24962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:50.659480 24962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:50.659523 24962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:50.663643 24962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:50.663707 24962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:50.663800 24962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:47:50.670019 24962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:45433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:33131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=39051
--tserver_master_addrs=127.7.171.62:34679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:47:50.671248 24962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:50.672446 24962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:50.683599 24968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:50.683642 24962 server_base.cc:1034] running on GCE node
W20260809 04:47:50.683602 24967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:50.683599 24970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:50.684329 24962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:50.685101 24962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:50.686306 24962 hybrid_clock.cc:648] HybridClock initialized: now 1786250870686268 us; error 48 us; skew 500 ppm
Aug 09 04:47:50 dist-test-slave-1zqn krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:50.689635 24962 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:47:50.690534 24962 webserver.cc:460] Webserver started at http://127.7.171.3:39051/ using document root <none> and password file <none>
I20260809 04:47:50.691160 24962 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:50.691236 24962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:50.695714 24962 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:47:50.697997 24978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:50.699275 24962 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:47:50.699417 24962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "2a4acd726da84c088082e33d3af47c87"
format_stamp: "Formatted at 2026-08-09 04:47:48 on dist-test-slave-1zqn"
server_key: "3fff5f4ba32a4bfdc06e0c6a6aec2a37"
server_key_iv: "7dd2c4eb21cf2dd51d41f1e630a21a51"
server_key_version: "encryptionkey@0"
I20260809 04:47:50.699985 24962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:50.720158 24962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:50.723752 24962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:50.723979 24962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:50.724704 24962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:50.725580 24962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:50.725653 24962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:50.725729 24962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:50.725772 24962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:50.735368 24962 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:33131
I20260809 04:47:50.735399 25102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:33131 every 8 connection(s)
I20260809 04:47:50.736426 24962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:47:50.740545 25105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:50.738738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:43997)
0809 04:47:50.739222 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:50.739331 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":134,"threads_started":1}
W20260809 04:47:50.741068 25103 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:34679 (0 consecutive failures): Network error: Failed to ping master at 127.7.171.62:34679: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
I20260809 04:47:50.741662 25105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:50.741447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:49655)
0809 04:47:50.741554 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:50.741603 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 04:47:50.742414 25105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:50.742225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:47469)
0809 04:47:50.742323 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:50.742365 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 04:47:50.742614 25103 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 04:47:50.743528 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 24962
I20260809 04:47:51.390700 25107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:51.390262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:44497)
0809 04:47:51.390522 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:51.390576 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":104,"threads_started":1}
I20260809 04:47:51.544894 25108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:51.544484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:35251)
0809 04:47:51.544741 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:51.544793 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":96,"threads_started":1}
I20260809 04:47:51.743602 25110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:51.743168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:50757)
0809 04:47:51.743438 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:51.743495 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":78,"threads_started":1}
I20260809 04:47:52.203147 25111 raft_consensus.cc:482] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:52.203383 25111 raft_consensus.cc:504] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:52.205050 25111 leader_election.cc:288] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131), 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825)
Aug 09 04:47:52 dist-test-slave-1zqn krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.2@KRBTEST.COM
Aug 09 04:47:52 dist-test-slave-1zqn krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250870, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.3@KRBTEST.COM
I20260809 04:47:52.222235 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:52.205632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:35825 (local address 127.7.171.1:60273)
0809 04:47:52.205963 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:52.205976 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:52.206080 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:52.207971 (+ 1891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:52.207974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:52.208379 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:52.209320 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:52.209345 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:52.211591 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:52.211595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:52.213244 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:52.213256 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:52.214310 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:52.214313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:52.214453 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:52.215179 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:52.215205 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:52.217194 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:52.219645 (+ 2451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:52.219655 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:52.219671 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:52.220094 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:52.220445 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:52.220448 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:52.220450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:52.220608 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:52.221264 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:52.221271 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:52.221510 (+ 239us) client_negotiation.cc:769] Sending connection context
0809 04:47:52.221695 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 04:47:52.222034 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":225,"spinlock_wait_cycles":32000,"thread_start_us":73,"threads_started":1}
I20260809 04:47:52.222275 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:52.205350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:56769 (local address 127.7.171.3:33131)
0809 04:47:52.205664 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:52.205697 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:52.205747 (+ 50us) server_negotiation.cc:407] Connection header received
0809 04:47:52.206778 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:52.206788 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:52.207295 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:52.207697 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:52.209568 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:52.211398 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:52.213697 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:52.214610 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:52.214791 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:52.217732 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:52.217737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:52.217749 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:52.217778 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:52.219622 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:52.220196 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:52.220208 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:52.220213 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:52.220332 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:52.220707 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:52.220710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:52.220712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:52.221135 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:52.221285 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:52.221837 (+ 552us) server_negotiation.cc:299] Negotiation successful
0809 04:47:52.222101 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":113,"threads_started":1}
I20260809 04:47:52.222327 25115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:52.205955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:60273 (local address 127.7.171.2:35825)
0809 04:47:52.206278 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:52.206292 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:52.206335 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:47:52.207248 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:52.207257 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:52.207697 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:52.208023 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:52.209568 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:52.211397 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:52.213387 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:52.214179 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:52.214369 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:52.217469 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:52.217477 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:52.217496 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:52.217531 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:52.219505 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:52.220209 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:52.220214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:52.220218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:52.220331 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:52.220749 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:52.220752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:52.220753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:52.221141 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:52.221285 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:52.221858 (+ 573us) server_negotiation.cc:299] Negotiation successful
0809 04:47:52.222134 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":79,"threads_started":1}
I20260809 04:47:52.222343 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:52.205209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:33131 (local address 127.7.171.1:56769)
0809 04:47:52.205484 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:52.205500 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:52.205606 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:52.207734 (+ 2128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:52.207739 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:52.208379 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:52.209320 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:52.209345 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:52.211591 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:52.211595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:52.213244 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:52.213256 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:52.214741 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:52.214748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:52.214823 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:52.215238 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:52.215250 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:52.217536 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:52.219746 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:52.219748 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:52.219750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:52.220035 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:52.220437 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:52.220441 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:52.220442 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:52.220600 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:52.221262 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:52.221268 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:52.221510 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 04:47:52.221665 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 04:47:52.222134 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":517,"spinlock_wait_cycles":9088,"thread_start_us":110,"threads_started":1}
W20260809 04:47:52.225657 24691 leader_election.cc:341] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131): Not found: Tablet not found: 5136603a0ca449ba9c720e0ab8be229f
W20260809 04:47:52.226279 24691 leader_election.cc:341] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825): Not found: Tablet not found: 5136603a0ca449ba9c720e0ab8be229f
I20260809 04:47:52.226357 24691 leader_election.cc:302] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9ef5af632ba244d98bbea049cb2cdf31; no voters: 2a4acd726da84c088082e33d3af47c87, 98bcb0a3fa254657b2179f217677d759
I20260809 04:47:52.226598 25111 raft_consensus.cc:2726] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:47:52.391741 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:52.391450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:55509)
0809 04:47:52.391593 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:52.391649 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 04:47:52.546255 25116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:52.545802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:58159)
0809 04:47:52.546075 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:52.546131 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":94,"threads_started":1}
I20260809 04:47:52.744979 25117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:52.744563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:48761)
0809 04:47:52.744823 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:52.744883 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":78,"threads_started":1}
I20260809 04:47:53.392992 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:53.392572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:45803)
0809 04:47:53.392838 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:53.392894 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":75,"threads_started":1}
I20260809 04:47:53.547389 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:53.546983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:57765)
0809 04:47:53.547236 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:53.547290 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260809 04:47:53.746282 25120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:53.745835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:47873)
0809 04:47:53.746103 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:53.746167 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
I20260809 04:47:53.949115 25121 raft_consensus.cc:482] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:53.949294 25121 raft_consensus.cc:504] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:53.949738 25121 leader_election.cc:288] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131), 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825)
W20260809 04:47:53.950678 24691 leader_election.cc:341] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825): Not found: Tablet not found: 5136603a0ca449ba9c720e0ab8be229f
W20260809 04:47:53.950804 24691 leader_election.cc:341] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131): Not found: Tablet not found: 5136603a0ca449ba9c720e0ab8be229f
I20260809 04:47:53.950837 24691 leader_election.cc:302] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9ef5af632ba244d98bbea049cb2cdf31; no voters: 2a4acd726da84c088082e33d3af47c87, 98bcb0a3fa254657b2179f217677d759
I20260809 04:47:53.950994 25121 raft_consensus.cc:2726] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:47:54.394178 25122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:54.393750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:34069)
0809 04:47:54.394002 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:54.394060 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":80,"threads_started":1}
I20260809 04:47:54.548748 25125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:54.548293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:48089)
0809 04:47:54.548564 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:54.548630 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
I20260809 04:47:54.747601 25126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:54.747155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:57379)
0809 04:47:54.747433 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:54.747495 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":83,"threads_started":1}
I20260809 04:47:55.395752 25127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:55.395237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:58695)
0809 04:47:55.395540 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:55.395612 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":81,"threads_started":1}
I20260809 04:47:55.549983 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:55.549533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:32837)
0809 04:47:55.549835 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:55.549888 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":135,"threads_started":1}
I20260809 04:47:55.748889 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:55.748459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:50019)
0809 04:47:55.748737 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:55.748794 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":100,"threads_started":1}
I20260809 04:47:56.397147 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:56.396698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:57765)
0809 04:47:56.396972 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:56.397044 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":107,"threads_started":1}
I20260809 04:47:56.551421 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:56.550847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:51261)
0809 04:47:56.551247 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:56.551314 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":198,"threads_started":1}
I20260809 04:47:56.750408 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:56.749887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:32801)
0809 04:47:56.750215 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:56.750276 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":128,"threads_started":1}
I20260809 04:47:57.218482 25133 raft_consensus.cc:482] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:57.218622 25133 raft_consensus.cc:504] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:47:57.219014 25133 leader_election.cc:288] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131), 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825)
W20260809 04:47:57.220036 24691 leader_election.cc:341] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825): Not found: Tablet not found: 5136603a0ca449ba9c720e0ab8be229f
W20260809 04:47:57.220266 24691 leader_election.cc:341] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131): Not found: Tablet not found: 5136603a0ca449ba9c720e0ab8be229f
I20260809 04:47:57.220331 24691 leader_election.cc:302] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9ef5af632ba244d98bbea049cb2cdf31; no voters: 2a4acd726da84c088082e33d3af47c87, 98bcb0a3fa254657b2179f217677d759
I20260809 04:47:57.220523 25133 raft_consensus.cc:2726] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:47:57.398331 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:57.397902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:52989)
0809 04:47:57.398176 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:57.398231 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":98,"threads_started":1}
I20260809 04:47:57.552927 25136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:57.552405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:49149)
0809 04:47:57.552758 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:57.552825 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":95,"threads_started":1}
I20260809 04:47:57.751834 25137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:57.751320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:57419)
0809 04:47:57.751660 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:57.751727 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":100,"threads_started":1}
I20260809 04:47:58.399585 25138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:58.399150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:39041)
0809 04:47:58.399415 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:58.399475 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
I20260809 04:47:58.554260 25139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:58.553800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:54361)
0809 04:47:58.554088 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:58.554157 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":86,"threads_started":1}
I20260809 04:47:58.753098 25140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:58.752683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:47935)
0809 04:47:58.752943 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:58.752998 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":85,"threads_started":1}
I20260809 04:47:59.400985 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:59.400555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:54295)
0809 04:47:59.400828 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:59.400883 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":99,"threads_started":1}
I20260809 04:47:59.555626 25143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:59.555145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:58973)
0809 04:47:59.555453 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:59.555515 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":89,"threads_started":1}
I20260809 04:47:59.754326 25144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:59.753922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:40383)
0809 04:47:59.754177 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:59.754232 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":82,"threads_started":1}
I20260809 04:48:00.402372 25145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:00.401948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:56793)
0809 04:48:00.402208 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:00.402269 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":82,"threads_started":1}
I20260809 04:48:00.557111 25146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:00.556586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:33201)
0809 04:48:00.556933 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:00.557000 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":119,"threads_started":1}
I20260809 04:48:00.755717 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:00.755222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:35493)
0809 04:48:00.755537 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:00.755611 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":104,"threads_started":1}
I20260809 04:48:01.403859 25148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:01.403253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:34647)
0809 04:48:01.403641 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:01.403726 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":117,"threads_started":1}
I20260809 04:48:01.558395 25149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:01.557966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:43823)
0809 04:48:01.558232 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:01.558294 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":83,"threads_started":1}
I20260809 04:48:01.717917 25151 raft_consensus.cc:482] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:01.718065 25151 raft_consensus.cc:504] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a4acd726da84c088082e33d3af47c87" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 33131 } } peers { permanent_uuid: "98bcb0a3fa254657b2179f217677d759" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 35825 } } peers { permanent_uuid: "9ef5af632ba244d98bbea049cb2cdf31" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 37385 } }
I20260809 04:48:01.718449 25151 leader_election.cc:288] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131), 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825)
W20260809 04:48:01.719388 24691 leader_election.cc:341] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a4acd726da84c088082e33d3af47c87 (127.7.171.3:33131): Not found: Tablet not found: 5136603a0ca449ba9c720e0ab8be229f
W20260809 04:48:01.719563 24691 leader_election.cc:341] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98bcb0a3fa254657b2179f217677d759 (127.7.171.2:35825): Not found: Tablet not found: 5136603a0ca449ba9c720e0ab8be229f
I20260809 04:48:01.719652 24691 leader_election.cc:302] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9ef5af632ba244d98bbea049cb2cdf31; no voters: 2a4acd726da84c088082e33d3af47c87, 98bcb0a3fa254657b2179f217677d759
I20260809 04:48:01.719869 25151 raft_consensus.cc:2726] T 5136603a0ca449ba9c720e0ab8be229f P 9ef5af632ba244d98bbea049cb2cdf31 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:48:01.757051 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:01.756559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:41491)
0809 04:48:01.756882 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:01.756951 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":78,"threads_started":1}
I20260809 04:48:02.405376 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:02.404834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:45673)
0809 04:48:02.405167 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:02.405247 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":80,"threads_started":1}
I20260809 04:48:02.559849 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:02.559270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:58049)
0809 04:48:02.559615 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:02.559721 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":98,"threads_started":1}
I20260809 04:48:02.758437 25155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:02.757965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:45021)
0809 04:48:02.758257 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:02.758325 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":88,"threads_started":1}
I20260809 04:48:03.406833 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:03.406270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:45643)
0809 04:48:03.406617 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:03.406700 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":107,"threads_started":1}
I20260809 04:48:03.561082 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:03.560683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:54313)
0809 04:48:03.560927 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:03.560981 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":79,"threads_started":1}
I20260809 04:48:03.759722 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:03.759254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:37535)
0809 04:48:03.759554 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:03.759621 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":85,"threads_started":1}
I20260809 04:48:04.408036 25159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:04.407609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:50549)
0809 04:48:04.407884 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:04.407938 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260809 04:48:04.562307 25160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:04.561883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:42965)
0809 04:48:04.562153 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:04.562211 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":81,"threads_started":1}
I20260809 04:48:04.761004 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:04.760566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:43075)
0809 04:48:04.760845 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:04.760905 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":90,"threads_started":1}
I20260809 04:48:05.409577 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:05.409154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.1:46035)
0809 04:48:05.409418 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:05.409476 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
I20260809 04:48:05.563644 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:05.563227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.2:50983)
0809 04:48:05.563477 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:05.563535 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260809 04:48:05.754010 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 24635
I20260809 04:48:05.762527 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:05.762075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34679 (local address 127.7.171.3:53765)
0809 04:48:05.762366 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:05.762428 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":100,"threads_started":1}
I20260809 04:48:05.764336 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 24818
I20260809 04:48:05.771775 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 24962
2026-08-09T04:48:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19290 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:05 dist-test-slave-1zqn krb5kdc[25167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:05 dist-test-slave-1zqn krb5kdc[25167](info): set up 2 sockets
Aug 09 04:48:05 dist-test-slave-1zqn krb5kdc[25167](info): commencing operation
krb5kdc: starting...
W20260809 04:48:06.926476 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:48:07 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:48:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:48:07.067188 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34927
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:34927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:07.188391 25181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:07.188716 25181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:07.188787 25181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:07.192943 25181 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 04:48:07.193035 25181 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 04:48:07.193059 25181 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 04:48:07.193080 25181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:07.193127 25181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:07.193168 25181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:07.193204 25181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:07.193240 25181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:07.198729 25181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:34927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:07.199875 25181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:07.200985 25181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:07.209923 25187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:07.209972 25186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:07.210016 25181 server_base.cc:1034] running on GCE node
W20260809 04:48:07.210222 25189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:07.210808 25181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:07.211906 25181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:07.213151 25181 hybrid_clock.cc:648] HybridClock initialized: now 1786250887213131 us; error 32 us; skew 500 ppm
Aug 09 04:48:07 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:07.219756 25181 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:48:07.220808 25181 webserver.cc:460] Webserver started at http://127.7.171.62:34499/ using document root <none> and password file <none>
I20260809 04:48:07.221444 25181 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:07.221511 25181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:07.221742 25181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:07.223953 25181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "56576a93975b4e8196f83720d69136bc"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "2b2fb5250c57a13c4cf632f680ad91b7"
server_key_iv: "5a6b35a2691cf2b3f523bd3478af6aad"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.224610 25181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "56576a93975b4e8196f83720d69136bc"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "2b2fb5250c57a13c4cf632f680ad91b7"
server_key_iv: "5a6b35a2691cf2b3f523bd3478af6aad"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.228636 25181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:48:07.231073 25196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:07.232369 25181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:48:07.232518 25181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "56576a93975b4e8196f83720d69136bc"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "2b2fb5250c57a13c4cf632f680ad91b7"
server_key_iv: "5a6b35a2691cf2b3f523bd3478af6aad"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.232635 25181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:07.247401 25181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:07.250327 25181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:07.250550 25181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:07.257737 25181 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:34927
I20260809 04:48:07.258035 25256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:34927 every 8 connection(s)
I20260809 04:48:07.258752 25181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:07.261602 25257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:07.265007 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 25181
I20260809 04:48:07.265161 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:48:07.265450 7852 external_mini_cluster.cc:1428] Setting key 01059f0f267d8b1666dc18dcaa87bb9d
I20260809 04:48:07.268131 25257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Bootstrap starting.
I20260809 04:48:07.270931 25257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:07.271792 25257 log.cc:824] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:07.274154 25257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: No bootstrap required, opened a new log
Aug 09 04:48:07 dist-test-slave-1zqn krb5kdc[25167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:48:07.278429 25257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:07.278762 25257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:07.278827 25257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56576a93975b4e8196f83720d69136bc, State: Initialized, Role: FOLLOWER
I20260809 04:48:07.279415 25257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:07.279594 25257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:07.279667 25257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:07.279850 25257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:07.280856 25257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:07.281288 25257 leader_election.cc:302] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56576a93975b4e8196f83720d69136bc; no voters:
I20260809 04:48:07.281667 25257 leader_election.cc:288] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:07.281746 25262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:07.281953 25262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 1 LEADER]: Becoming Leader. State: Replica: 56576a93975b4e8196f83720d69136bc, State: Running, Role: LEADER
I20260809 04:48:07.282269 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:07.266840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.7.171.62:34927)
0809 04:48:07.267299 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:07.267319 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:07.267361 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:48:07.268255 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:07.268279 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:07.268721 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:07.269089 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:07.269839 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:07.271313 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:07.272313 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:07.272886 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:07.273041 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:07.275668 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:07.275674 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:07.275694 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:07.275728 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:07.278470 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:07.279116 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:07.279124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:07.279132 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:07.279277 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:07.279631 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:07.279635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:07.279637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:07.280269 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:07.280503 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:07.280840 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 04:48:07.281168 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":127,"threads_started":1}
I20260809 04:48:07.282325 25262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:07.283056 25257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:07.284469 25264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56576a93975b4e8196f83720d69136bc. Latest consensus state: current_term: 1 leader_uuid: "56576a93975b4e8196f83720d69136bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } } }
I20260809 04:48:07.284897 25264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:07.284770 25263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56576a93975b4e8196f83720d69136bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } } }
I20260809 04:48:07.285248 25263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: This master's current role is: LEADER
W20260809 04:48:07.288012 25277 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:48:07.288160 25277 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:48:07.288264 25275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:07.289211 25275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:07.295082 25275 catalog_manager.cc:1347] Generated new cluster ID: d186f7eb702a42a88162c825243bc9a6
I20260809 04:48:07.295166 25275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:07.314850 25275 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:07.316272 25275 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:07.332118 25275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Generated new TSK 0
I20260809 04:48:07.333041 25275 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:48:07.365257 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34927
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:07.481369 25283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:07.481648 25283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:07.481693 25283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:07.486150 25283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:07.486223 25283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:07.486323 25283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:48:07.491832 25283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:07.493037 25283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:07.494148 25283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:07.504801 25293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:07.504906 25291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:07.504801 25290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:07.505014 25283 server_base.cc:1034] running on GCE node
I20260809 04:48:07.505584 25283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:07.506244 25283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:07.507459 25283 hybrid_clock.cc:648] HybridClock initialized: now 1786250887507442 us; error 27 us; skew 500 ppm
Aug 09 04:48:07 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:07.510490 25283 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:48:07.511372 25283 webserver.cc:460] Webserver started at http://127.7.171.1:40067/ using document root <none> and password file <none>
I20260809 04:48:07.511927 25283 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:07.511988 25283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:07.512244 25283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:07.514320 25283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "a62e712863fb948205cd886e5d4ad92a"
server_key_iv: "c27635c158a09ffb569072bc24930a4e"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.514918 25283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "a62e712863fb948205cd886e5d4ad92a"
server_key_iv: "c27635c158a09ffb569072bc24930a4e"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.518708 25283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:48:07.521308 25300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:07.522589 25283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:48:07.522755 25283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "a62e712863fb948205cd886e5d4ad92a"
server_key_iv: "c27635c158a09ffb569072bc24930a4e"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.522876 25283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:07.541258 25283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:07.544925 25283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:07.545178 25283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:07.545868 25283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:07.546759 25283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:07.546834 25283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:07.546909 25283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:07.546962 25283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:07.557797 25283 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:36525
I20260809 04:48:07.557824 25424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:36525 every 8 connection(s)
I20260809 04:48:07.558830 25283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:48:07.563139 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 25283
I20260809 04:48:07.563295 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:48:07.563676 7852 external_mini_cluster.cc:1428] Setting key 8c045b0249d1bea82fe7a2447760f300
Aug 09 04:48:07 dist-test-slave-1zqn krb5kdc[25167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:48:07.574493 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:07.560841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:59307 (local address 127.7.171.62:34927)
0809 04:48:07.561048 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:07.561054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:07.562197 (+ 1143us) server_negotiation.cc:407] Connection header received
0809 04:48:07.563108 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:07.563114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:07.563182 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:07.563480 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:07.564838 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:07.565506 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:07.566683 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:07.567115 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:07.567248 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:07.570152 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:07.570160 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:07.570166 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:07.570202 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:07.571936 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:07.572691 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:07.572694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:07.572696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:07.572784 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:07.573242 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:07.573245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:07.573247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:07.573453 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:07.573536 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:07.574220 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 04:48:07.574340 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:48:07.575695 25427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:07.561338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:59307)
0809 04:48:07.562031 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:07.562074 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:07.562878 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:07.563514 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:07.563521 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:07.564005 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:07.564672 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:07.564689 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:07.565654 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:07.565656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:07.566536 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:07.566545 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:07.567276 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:07.567279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:07.567424 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:07.568135 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:07.568152 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:07.569957 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:07.572116 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:07.572153 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:07.572173 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:07.572578 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:07.572906 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:07.572911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:07.572914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:07.573108 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:07.573573 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:07.573581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:07.573935 (+ 354us) client_negotiation.cc:769] Sending connection context
0809 04:48:07.574173 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 04:48:07.574559 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":118,"threads_started":1}
I20260809 04:48:07.577162 25425 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34927
I20260809 04:48:07.577438 25425 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:07.577958 25425 heartbeater.cc:511] Master 127.7.171.62:34927 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
I20260809 04:48:07.579932 25216 ts_manager.cc:194] Registered new tserver with Master: 262a6b80c3fc47bc8a752ed6c9a1b4cb (127.7.171.1:36525)
I20260809 04:48:07.581369 25216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:59307
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:48:07.598976 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34927
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:07.714437 25430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:07.714710 25430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:07.714754 25430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:07.718837 25430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:07.718911 25430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:07.718977 25430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:48:07.724454 25430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:07.725457 25430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:07.726517 25430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:48:07.736514 25430 server_base.cc:1034] running on GCE node
W20260809 04:48:07.736485 25436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:07.736462 25437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:07.736711 25439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:07.737129 25430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:07.737833 25430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:07.739019 25430 hybrid_clock.cc:648] HybridClock initialized: now 1786250887738990 us; error 40 us; skew 500 ppm
Aug 09 04:48:07 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:07.741911 25430 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:48:07.742882 25430 webserver.cc:460] Webserver started at http://127.7.171.2:41617/ using document root <none> and password file <none>
I20260809 04:48:07.743462 25430 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:07.743556 25430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:07.743767 25430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:07.745960 25430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "5ea5833ff17d42f5b76c082fe8f99413"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "7c1a04306136ca6b4370495a992804dd"
server_key_iv: "a832d02b71676b05dc3cb4271c0c124c"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.746543 25430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "5ea5833ff17d42f5b76c082fe8f99413"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "7c1a04306136ca6b4370495a992804dd"
server_key_iv: "a832d02b71676b05dc3cb4271c0c124c"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.750494 25430 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:48:07.753217 25446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:07.754474 25430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:48:07.754734 25430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "5ea5833ff17d42f5b76c082fe8f99413"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "7c1a04306136ca6b4370495a992804dd"
server_key_iv: "a832d02b71676b05dc3cb4271c0c124c"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.754864 25430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:07.773695 25430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:07.776966 25430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:07.777202 25430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:07.777858 25430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:07.778769 25430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:07.778846 25430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:07.778921 25430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:07.778967 25430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:07.789371 25566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:45883 every 8 connection(s)
I20260809 04:48:07.789474 25430 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:45883
I20260809 04:48:07.790426 25430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:48:07.797322 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 25430
I20260809 04:48:07.797457 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:48:07.797851 7852 external_mini_cluster.cc:1428] Setting key 56302e1a4b1ce041695a6370b3022ef7
Aug 09 04:48:07 dist-test-slave-1zqn krb5kdc[25167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:48:07.807930 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:07.792628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:46825 (local address 127.7.171.62:34927)
0809 04:48:07.792777 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:07.792781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:07.793787 (+ 1006us) server_negotiation.cc:407] Connection header received
0809 04:48:07.794932 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:07.794936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:07.794996 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:07.795115 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:07.796697 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:07.797679 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:07.799243 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:07.799691 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:07.799838 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:07.803016 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:07.803020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:07.803022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:07.803055 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:07.804827 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:07.805426 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:07.805429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:07.805431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:07.805523 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:07.805996 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:07.806002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:07.806007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:07.806287 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:07.806401 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:07.807477 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0809 04:48:07.807649 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:48:07.809974 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:07.792927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:46825)
0809 04:48:07.793633 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:07.793677 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:07.794703 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:07.795297 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:07.795304 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:07.795704 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:07.796530 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:07.796551 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:07.797843 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:07.797847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:07.799077 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:07.799088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:07.799904 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:07.799909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:07.800198 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:07.800981 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:07.801001 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:07.802839 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:07.804958 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:07.804964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:07.804980 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:07.805326 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:07.805637 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:07.805644 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:07.805650 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:07.805867 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:07.806430 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:07.806440 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:07.807124 (+ 684us) client_negotiation.cc:769] Sending connection context
0809 04:48:07.808251 (+ 1127us) client_negotiation.cc:240] Negotiation successful
0809 04:48:07.808715 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":127,"threads_started":1}
I20260809 04:48:07.811503 25567 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34927
I20260809 04:48:07.811784 25567 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:48:07.812454 25567 heartbeater.cc:511] Master 127.7.171.62:34927 requested a full tablet report, sending...
I20260809 04:48:07.813589 25216 ts_manager.cc:194] Registered new tserver with Master: 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883)
I20260809 04:48:07.814638 25216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:46825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:48:07.833321 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34927
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:07.950681 25573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:07.950945 25573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:07.950999 25573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:07.955067 25573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:07.955128 25573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:07.955186 25573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:48:07.960654 25573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:07.961638 25573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:07.962615 25573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:07.972611 25582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:07.972711 25579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:07.972719 25580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:07.972836 25573 server_base.cc:1034] running on GCE node
I20260809 04:48:07.973291 25573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:07.973990 25573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:07.975175 25573 hybrid_clock.cc:648] HybridClock initialized: now 1786250887975162 us; error 26 us; skew 500 ppm
Aug 09 04:48:07 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:07.978108 25573 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:48:07.978978 25573 webserver.cc:460] Webserver started at http://127.7.171.3:36395/ using document root <none> and password file <none>
I20260809 04:48:07.979579 25573 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:07.979681 25573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:07.979923 25573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:07.982100 25573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "1c2e8a79ed2b46eaac833bf2161a258e"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "a7112678d823f37f5f645fe3ab4a1a22"
server_key_iv: "c0199cc97dc89038ec70f7948b66d8b4"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.982697 25573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "1c2e8a79ed2b46eaac833bf2161a258e"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "a7112678d823f37f5f645fe3ab4a1a22"
server_key_iv: "c0199cc97dc89038ec70f7948b66d8b4"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.986753 25573 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:48:07.989269 25591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:07.990551 25573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:48:07.990687 25573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "1c2e8a79ed2b46eaac833bf2161a258e"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "a7112678d823f37f5f645fe3ab4a1a22"
server_key_iv: "c0199cc97dc89038ec70f7948b66d8b4"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.990800 25573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:08.014686 25573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:08.017927 25573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:08.018137 25573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:08.018760 25573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:08.019611 25573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:08.019682 25573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:08.019769 25573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:08.019819 25573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:08.029196 25573 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:45159
I20260809 04:48:08.029223 25714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:45159 every 8 connection(s)
I20260809 04:48:08.030104 25573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:48:08.031241 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 25573
I20260809 04:48:08.031380 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:48:08.031646 7852 external_mini_cluster.cc:1428] Setting key 8d3b0c52f209d955754e75c981603008
Aug 09 04:48:08 dist-test-slave-1zqn krb5kdc[25167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250887, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:48:08.044448 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.032002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:54219 (local address 127.7.171.62:34927)
0809 04:48:08.032203 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:08.032209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:08.033486 (+ 1277us) server_negotiation.cc:407] Connection header received
0809 04:48:08.034523 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:08.034526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:08.034589 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:08.034665 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:08.035819 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.036454 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.037610 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.037837 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.037943 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.040726 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:08.040730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:08.040732 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:08.040760 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:08.042236 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:08.042794 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:08.042798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:08.042800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:08.042887 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:08.043262 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:08.043265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:08.043267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:08.043460 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:08.043567 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:08.044135 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 04:48:08.044273 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:48:08.045511 25718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.032514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:54219)
0809 04:48:08.033350 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.033394 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:08.034324 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:08.034834 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:08.034841 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:08.035205 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:08.035679 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.035693 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.036585 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.036595 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.037474 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.037483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.038000 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.038004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.038185 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.038810 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:08.038826 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:08.040510 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:08.042372 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:08.042379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:08.042397 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:08.042683 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:08.042987 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:08.042991 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:08.042993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:08.043150 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:08.043582 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:08.043590 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:08.043853 (+ 263us) client_negotiation.cc:769] Sending connection context
0809 04:48:08.044050 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 04:48:08.044464 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":140,"threads_started":1}
I20260809 04:48:08.046916 25715 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34927
I20260809 04:48:08.047195 25715 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:08.047709 25715 heartbeater.cc:511] Master 127.7.171.62:34927 requested a full tablet report, sending...
I20260809 04:48:08.048933 25216 ts_manager.cc:194] Registered new tserver with Master: 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159)
I20260809 04:48:08.049515 25216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:54219
I20260809 04:48:08.058045 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:48:08.067847 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.060366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.7.171.62:34927)
0809 04:48:08.060513 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:08.060517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:08.060689 (+ 172us) server_negotiation.cc:407] Connection header received
0809 04:48:08.060789 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:08.060791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:08.060832 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:08.060896 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:08.061570 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.062077 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.062873 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.063077 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.063154 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.064550 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:08.064554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:08.064557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:08.064591 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:08.066109 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:08.066753 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:08.066757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:08.066759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:08.066853 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:08.067101 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:08.067104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:08.067105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:08.067325 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:08.067433 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:08.067620 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 04:48:08.067722 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:48:08.070739 25216 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:48:08.074112 25216 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:48:08.089496 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.083445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45159 (local address 127.0.0.1:54468)
0809 04:48:08.084415 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.084429 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:08.084515 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:08.085267 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:08.085270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:08.085299 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:08.085468 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.085475 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.086496 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.086501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.087966 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.087979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.088876 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.088880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.088998 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.089038 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 04:48:08.089181 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 04:48:08.089318 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"mutex_wait_us":133,"spinlock_wait_cycles":768,"thread_start_us":101,"threads_started":1}
I20260809 04:48:08.089644 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.083865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57244 (local address 127.7.171.2:45883)
0809 04:48:08.084428 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:08.084442 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:08.084470 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:48:08.084574 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:08.084583 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:08.084851 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:08.085079 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:08.085415 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.086673 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.088231 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.089051 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.089176 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.089318 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:08.089380 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 04:48:08.089482 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":111,"threads_started":1}
I20260809 04:48:08.090006 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.083069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36525 (local address 127.0.0.1:33298)
0809 04:48:08.083838 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.083891 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:08.084126 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:08.085276 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:08.085279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:08.085301 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:08.085468 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.085475 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.086858 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.086861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.087996 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.088006 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.089550 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.089554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.089656 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.089681 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:48:08.089754 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 04:48:08.089847 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"mutex_wait_us":58,"spinlock_wait_cycles":3712,"thread_start_us":70,"threads_started":1}
I20260809 04:48:08.090144 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.083873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54468 (local address 127.7.171.3:45159)
0809 04:48:08.084475 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:08.084495 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:08.084518 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:48:08.084684 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:08.084701 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:08.084850 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:08.085102 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:08.085628 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.086360 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.088137 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.088740 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.088872 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.089032 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:08.089408 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 04:48:08.089499 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":204,"threads_started":1}
I20260809 04:48:08.089499 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.083299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:45883 (local address 127.0.0.1:57244)
0809 04:48:08.083838 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.083926 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:08.084102 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:08.085062 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:08.085068 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:08.085090 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:08.085273 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.085281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.086819 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.086822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.088089 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.088098 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.089182 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.089186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.089269 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.089289 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:48:08.089348 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 04:48:08.089400 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":54,"thread_start_us":121,"threads_started":1}
I20260809 04:48:08.090391 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.083182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33298 (local address 127.7.171.1:36525)
0809 04:48:08.084741 (+ 1559us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:08.084747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:08.084771 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:48:08.084856 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:08.084864 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:08.085073 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:08.085211 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:08.085594 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.086729 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.088197 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.089051 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.089176 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.089318 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:08.089960 (+ 642us) server_negotiation.cc:299] Negotiation successful
0809 04:48:08.090074 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1456,"thread_start_us":116,"threads_started":1}
I20260809 04:48:08.093050 25497 tablet_service.cc:1467] Processing CreateTablet for tablet cb4c06635b544f7c81cfcb477915e619 (DEFAULT_TABLE table=test-table [id=39476632fd1343b9b39997f4b9b96cfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:08.093142 25351 tablet_service.cc:1467] Processing CreateTablet for tablet cb4c06635b544f7c81cfcb477915e619 (DEFAULT_TABLE table=test-table [id=39476632fd1343b9b39997f4b9b96cfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:08.093132 25640 tablet_service.cc:1467] Processing CreateTablet for tablet cb4c06635b544f7c81cfcb477915e619 (DEFAULT_TABLE table=test-table [id=39476632fd1343b9b39997f4b9b96cfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:08.094096 25497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4c06635b544f7c81cfcb477915e619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:08.094172 25351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4c06635b544f7c81cfcb477915e619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:08.094233 25640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb4c06635b544f7c81cfcb477915e619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:08.100716 25734 tablet_bootstrap.cc:492] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: Bootstrap starting.
I20260809 04:48:08.102275 25736 tablet_bootstrap.cc:492] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: Bootstrap starting.
I20260809 04:48:08.102416 25735 tablet_bootstrap.cc:492] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Bootstrap starting.
I20260809 04:48:08.103557 25734 tablet_bootstrap.cc:654] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:08.104377 25734 log.cc:824] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:08.104494 25736 tablet_bootstrap.cc:654] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:08.104494 25735 tablet_bootstrap.cc:654] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:08.105413 25735 log.cc:824] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:08.105705 25736 log.cc:824] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:08.106550 25734 tablet_bootstrap.cc:492] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: No bootstrap required, opened a new log
I20260809 04:48:08.106752 25734 ts_tablet_manager.cc:1397] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:48:08.107357 25735 tablet_bootstrap.cc:492] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: No bootstrap required, opened a new log
I20260809 04:48:08.107564 25735 ts_tablet_manager.cc:1397] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:48:08.107672 25736 tablet_bootstrap.cc:492] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: No bootstrap required, opened a new log
I20260809 04:48:08.107847 25736 ts_tablet_manager.cc:1397] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:48:08.110769 25734 raft_consensus.cc:348] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } }
I20260809 04:48:08.111063 25734 raft_consensus.cc:374] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:08.111123 25734 raft_consensus.cc:729] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ea5833ff17d42f5b76c082fe8f99413, State: Initialized, Role: FOLLOWER
I20260809 04:48:08.110947 25735 raft_consensus.cc:348] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } }
I20260809 04:48:08.111191 25735 raft_consensus.cc:374] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:08.111249 25735 raft_consensus.cc:729] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262a6b80c3fc47bc8a752ed6c9a1b4cb, State: Initialized, Role: FOLLOWER
I20260809 04:48:08.111629 25734 consensus_queue.cc:260] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } }
I20260809 04:48:08.111716 25735 consensus_queue.cc:260] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } }
I20260809 04:48:08.111758 25736 raft_consensus.cc:348] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } }
I20260809 04:48:08.112126 25736 raft_consensus.cc:374] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:08.112176 25736 raft_consensus.cc:729] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c2e8a79ed2b46eaac833bf2161a258e, State: Initialized, Role: FOLLOWER
I20260809 04:48:08.112548 25736 consensus_queue.cc:260] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } }
I20260809 04:48:08.112764 25735 ts_tablet_manager.cc:1428] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 04:48:08.112766 25734 ts_tablet_manager.cc:1428] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260809 04:48:08.113140 25567 heartbeater.cc:503] Master 127.7.171.62:34927 was elected leader, sending a full tablet report...
I20260809 04:48:08.113421 25736 ts_tablet_manager.cc:1428] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:48:08.113425 25715 heartbeater.cc:503] Master 127.7.171.62:34927 was elected leader, sending a full tablet report...
I20260809 04:48:08.113823 25425 heartbeater.cc:503] Master 127.7.171.62:34927 was elected leader, sending a full tablet report...
I20260809 04:48:08.249756 25744 raft_consensus.cc:482] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:08.250051 25744 raft_consensus.cc:504] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } }
I20260809 04:48:08.251539 25744 leader_election.cc:288] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262a6b80c3fc47bc8a752ed6c9a1b4cb (127.7.171.1:36525), 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883)
I20260809 04:48:08.255875 25718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.251959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:45883 (local address 127.7.171.3:38729)
0809 04:48:08.252101 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.252129 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:08.252239 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:08.252591 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:08.252593 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:08.252612 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:08.252734 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.252742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.253768 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.253774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.254883 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.254894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.255545 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.255548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.255627 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.255650 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:48:08.255707 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 04:48:08.255763 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":25}
I20260809 04:48:08.255930 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.252233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:38729 (local address 127.7.171.2:45883)
0809 04:48:08.252368 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:08.252373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:08.252387 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:48:08.252443 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:08.252445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:08.252484 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:08.252578 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:08.252888 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.253659 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.255011 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.255436 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.255541 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.255633 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:08.255734 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 04:48:08.255804 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:48:08.256641 25518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4c06635b544f7c81cfcb477915e619" candidate_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ea5833ff17d42f5b76c082fe8f99413" is_pre_election: true
I20260809 04:48:08.256750 25745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.252388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36525 (local address 127.7.171.3:51709)
0809 04:48:08.253031 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.253044 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:08.253154 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:08.253386 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:08.253389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:08.253405 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:08.253517 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.253522 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.254248 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.254251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.255300 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:08.255307 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.256313 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.256318 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:08.256420 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.256437 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 04:48:08.256529 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:48:08.256602 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":1,"thread_start_us":158,"threads_started":1}
I20260809 04:48:08.256773 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.252908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:51709 (local address 127.7.171.1:36525)
0809 04:48:08.253019 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:08.253023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:08.253104 (+ 81us) server_negotiation.cc:407] Connection header received
0809 04:48:08.253246 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:08.253248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:08.253292 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:08.253352 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:08.253643 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.254135 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.255420 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.256030 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.256178 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.256236 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:08.256547 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 04:48:08.256663 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 04:48:08.257025 25518 raft_consensus.cc:2454] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c2e8a79ed2b46eaac833bf2161a258e in term 0.
I20260809 04:48:08.257364 25373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4c06635b544f7c81cfcb477915e619" candidate_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" is_pre_election: true
I20260809 04:48:08.257535 25595 leader_election.cc:302] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c2e8a79ed2b46eaac833bf2161a258e, 5ea5833ff17d42f5b76c082fe8f99413; no voters:
I20260809 04:48:08.257706 25373 raft_consensus.cc:2454] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c2e8a79ed2b46eaac833bf2161a258e in term 0.
I20260809 04:48:08.257870 25744 raft_consensus.cc:2781] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:48:08.258009 25744 raft_consensus.cc:482] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:48:08.258143 25744 raft_consensus.cc:3037] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:08.259445 25744 raft_consensus.cc:504] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } }
I20260809 04:48:08.259882 25744 leader_election.cc:288] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [CANDIDATE]: Term 1 election: Requested vote from peers 262a6b80c3fc47bc8a752ed6c9a1b4cb (127.7.171.1:36525), 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883)
I20260809 04:48:08.260262 25373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4c06635b544f7c81cfcb477915e619" candidate_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb"
I20260809 04:48:08.260365 25518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb4c06635b544f7c81cfcb477915e619" candidate_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ea5833ff17d42f5b76c082fe8f99413"
I20260809 04:48:08.260394 25373 raft_consensus.cc:3037] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:08.260499 25518 raft_consensus.cc:3037] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:08.261771 25373 raft_consensus.cc:2454] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c2e8a79ed2b46eaac833bf2161a258e in term 1.
I20260809 04:48:08.261797 25518 raft_consensus.cc:2454] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c2e8a79ed2b46eaac833bf2161a258e in term 1.
I20260809 04:48:08.268249 25595 leader_election.cc:302] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c2e8a79ed2b46eaac833bf2161a258e, 5ea5833ff17d42f5b76c082fe8f99413; no voters:
I20260809 04:48:08.268455 25744 raft_consensus.cc:2781] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:08.268683 25744 raft_consensus.cc:686] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 1 LEADER]: Becoming Leader. State: Replica: 1c2e8a79ed2b46eaac833bf2161a258e, State: Running, Role: LEADER
I20260809 04:48:08.269033 25744 consensus_queue.cc:237] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } }
I20260809 04:48:08.272933 25214 catalog_manager.cc:5495] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3). New cstate: current_term: 1 leader_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } health_report { overall_health: HEALTHY } } }
W20260809 04:48:08.281030 25717 tablet.cc:2369] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:48:08.291712 25568 tablet.cc:2369] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:48:08.310310 25426 tablet.cc:2369] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:48:08.324977 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.321498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54482 (local address 127.7.171.3:45159)
0809 04:48:08.321664 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:08.321668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:08.321684 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:48:08.321765 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:08.321767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:08.321824 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:08.321931 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:48:08.322151 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.322663 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.323493 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:08.323708 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:08.323810 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:08.324145 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:48:08.324564 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:48:08.324681 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:08.324809 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 04:48:08.324863 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:48:08.331336 25518 raft_consensus.cc:1261] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413 [term 1 FOLLOWER]: Refusing update from remote peer 1c2e8a79ed2b46eaac833bf2161a258e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:08.331337 25373 raft_consensus.cc:1261] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Refusing update from remote peer 1c2e8a79ed2b46eaac833bf2161a258e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:08.332093 25746 consensus_queue.cc:1035] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:08.332274 25744 consensus_queue.cc:1035] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [LEADER]: Connected to new peer: Peer: permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:08.340560 25750 mvcc.cc:205] Tried to move back new op lower bound from 7316483638597029888 to 7316483638353186816. Current Snapshot: MvccSnapshot[applied={T|T < 7316483638597029888}]
I20260809 04:48:08.350658 25215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:08.350893 25215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:08.353770 25215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Sending DeleteTablet for 3 replicas of tablet cb4c06635b544f7c81cfcb477915e619
I20260809 04:48:08.354631 25497 tablet_service.cc:1514] Processing DeleteTablet for tablet cb4c06635b544f7c81cfcb477915e619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:08 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:57244
I20260809 04:48:08.354796 25640 tablet_service.cc:1514] Processing DeleteTablet for tablet cb4c06635b544f7c81cfcb477915e619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:08 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:54468
I20260809 04:48:08.355042 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 25283
I20260809 04:48:08.357276 25761 tablet_replica.cc:330] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: stopping tablet replica
I20260809 04:48:08.357718 25761 raft_consensus.cc:2229] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:48:08.358394 25761 raft_consensus.cc:2258] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:08.361709 25761 ts_tablet_manager.cc:1905] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:08.364389 25760 tablet_replica.cc:330] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: stopping tablet replica
I20260809 04:48:08.364792 25760 raft_consensus.cc:2229] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:08.365096 25761 ts_tablet_manager.cc:1918] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:08.365193 25760 raft_consensus.cc:2258] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:08.365302 25761 log.cc:1196] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/cb4c06635b544f7c81cfcb477915e619
I20260809 04:48:08.365767 25761 ts_tablet_manager.cc:1939] T cb4c06635b544f7c81cfcb477915e619 P 1c2e8a79ed2b46eaac833bf2161a258e: Deleting consensus metadata
I20260809 04:48:08.366954 25760 ts_tablet_manager.cc:1905] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:08.367319 25198 catalog_manager.cc:4841] TS 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159): tablet cb4c06635b544f7c81cfcb477915e619 (table test-table [id=39476632fd1343b9b39997f4b9b96cfe]) successfully deleted
W20260809 04:48:08.368196 25200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:36525 (error 108)
I20260809 04:48:08.368477 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 25430
I20260809 04:48:08.369570 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.369207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36525 (local address 127.0.0.1:33310)
0809 04:48:08.369404 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.369475 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:36525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260809 04:48:08.369855 25200 catalog_manager.cc:4584] TS 262a6b80c3fc47bc8a752ed6c9a1b4cb (127.7.171.1:36525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb4c06635b544f7c81cfcb477915e619: Network error: Client connection negotiation failed: client connection to 127.7.171.1:36525: connect: Connection refused (error 111)
I20260809 04:48:08.376698 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 25573
I20260809 04:48:08.376920 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.376594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:45883 (local address 127.0.0.1:57246)
0809 04:48:08.376750 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.376819 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.2:45883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 04:48:08.385689 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 25181
I20260809 04:48:28.394520 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34927
--webserver_interface=127.7.171.62
--webserver_port=34499
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:34927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:28.511965 25763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:28.512300 25763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:28.512341 25763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:28.516320 25763 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 04:48:28.516374 25763 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 04:48:28.516390 25763 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 04:48:28.516404 25763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:28.516422 25763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:28.516439 25763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:28.516495 25763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:28.516523 25763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:28.521889 25763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:34927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=34499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:28.522967 25763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:28.524106 25763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:28.538694 25773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:28.538764 25769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:28.538837 25771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:28.538753 25763 server_base.cc:1034] running on GCE node
I20260809 04:48:28.539674 25763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:28.540949 25763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:28.542129 25763 hybrid_clock.cc:648] HybridClock initialized: now 1786250908542110 us; error 27 us; skew 500 ppm
Aug 09 04:48:28 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250908, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:28.550416 25763 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:48:28.551749 25763 webserver.cc:460] Webserver started at http://127.7.171.62:34499/ using document root <none> and password file <none>
I20260809 04:48:28.552482 25763 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:28.552588 25763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:28.556829 25763 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:48:28.559229 25780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:28.560768 25763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:48:28.560925 25763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "56576a93975b4e8196f83720d69136bc"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "2b2fb5250c57a13c4cf632f680ad91b7"
server_key_iv: "5a6b35a2691cf2b3f523bd3478af6aad"
server_key_version: "encryptionkey@0"
I20260809 04:48:28.561620 25763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:28.587055 25763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:28.590965 25763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:28.591214 25763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:28.600168 25763 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:34927
I20260809 04:48:28.600492 25837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:34927 every 8 connection(s)
I20260809 04:48:28.601472 25763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:28.603195 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 25763
I20260809 04:48:28.603699 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:36525
--local_ip_for_outbound_sockets=127.7.171.1
--tserver_master_addrs=127.7.171.62:34927
--webserver_port=40067
--webserver_interface=127.7.171.1
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:48:28.606401 25838 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:48:28.608210 25838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Bootstrap starting.
I20260809 04:48:28.626775 25838 log.cc:824] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:28.635746 25838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:48:28.636274 25838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Bootstrap complete.
I20260809 04:48:28.640957 25838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:28.641414 25838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56576a93975b4e8196f83720d69136bc, State: Initialized, Role: FOLLOWER
I20260809 04:48:28.642000 25838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:28.642160 25838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:28.642266 25838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:28.642396 25838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:48:28.644547 25838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:28.644927 25838 leader_election.cc:302] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56576a93975b4e8196f83720d69136bc; no voters:
I20260809 04:48:28.645331 25838 leader_election.cc:288] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:48:28.645430 25842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:48:28.645674 25842 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 2 LEADER]: Becoming Leader. State: Replica: 56576a93975b4e8196f83720d69136bc, State: Running, Role: LEADER
I20260809 04:48:28.646066 25842 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:28.646747 25838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:28.647679 25843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "56576a93975b4e8196f83720d69136bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } } }
I20260809 04:48:28.647819 25843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:28.647744 25844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56576a93975b4e8196f83720d69136bc. Latest consensus state: current_term: 2 leader_uuid: "56576a93975b4e8196f83720d69136bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } } }
I20260809 04:48:28.647879 25844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:28.652205 25857 catalog_manager.cc:1259] Loaded cluster ID: d186f7eb702a42a88162c825243bc9a6
I20260809 04:48:28.652263 25857 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: loading cluster ID for follower catalog manager: success
I20260809 04:48:28.654162 25857 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: acquiring CA information for follower catalog manager: success
I20260809 04:48:28.655463 25857 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:48:28.655609 25850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:28.657837 25850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:28.658427 25850 catalog_manager.cc:1259] Loaded cluster ID: d186f7eb702a42a88162c825243bc9a6
I20260809 04:48:28.658490 25850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:28.659237 25850 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:28.659898 25850 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Loaded TSK: 0
I20260809 04:48:28.674889 25850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Generated new TSK 1
I20260809 04:48:28.675199 25850 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:48:28.743992 25840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:28.744411 25840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:28.744521 25840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:28.749130 25840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:28.749218 25840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:28.749322 25840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:48:28.755051 25840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:36525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=40067
--tserver_master_addrs=127.7.171.62:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:28.756292 25840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:28.757408 25840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:28.767935 25867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:28.767967 25866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:28.768181 25869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:28.768356 25840 server_base.cc:1034] running on GCE node
I20260809 04:48:28.768919 25840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:28.769793 25840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:28.771030 25840 hybrid_clock.cc:648] HybridClock initialized: now 1786250908770984 us; error 55 us; skew 500 ppm
Aug 09 04:48:28 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250908, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:28.774863 25840 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:48:28.776183 25840 webserver.cc:460] Webserver started at http://127.7.171.1:40067/ using document root <none> and password file <none>
I20260809 04:48:28.776988 25840 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:28.777112 25840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:28.782743 25840 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:48:28.785902 25876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:28.787657 25840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:48:28.787789 25840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "a62e712863fb948205cd886e5d4ad92a"
server_key_iv: "c27635c158a09ffb569072bc24930a4e"
server_key_version: "encryptionkey@0"
I20260809 04:48:28.788590 25840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:28.837483 25840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:28.841391 25840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:28.841635 25840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:28.842392 25840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:28.843724 25884 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 04:48:28.848549 25840 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 04:48:28.848626 25840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260809 04:48:28.848707 25840 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 04:48:28.850623 25840 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 04:48:28.850692 25840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 04:48:28.850975 25884 tablet_bootstrap.cc:492] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Bootstrap starting.
I20260809 04:48:28.863554 25840 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:36525
I20260809 04:48:28.863687 26003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:36525 every 8 connection(s)
I20260809 04:48:28.865080 25840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:48:28.873247 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 25840
I20260809 04:48:28.873813 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:45883
--local_ip_for_outbound_sockets=127.7.171.2
--tserver_master_addrs=127.7.171.62:34927
--webserver_port=41617
--webserver_interface=127.7.171.2
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:48:28.875113 25884 log.cc:824] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:48:28 dist-test-slave-1zqn krb5kdc[25167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250908, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:48:28.883841 25884 tablet_bootstrap.cc:492] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:48:28.884332 25884 tablet_bootstrap.cc:492] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Bootstrap complete.
I20260809 04:48:28.884766 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.867419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:58463 (local address 127.7.171.62:34927)
0809 04:48:28.867796 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:28.867818 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:28.868251 (+ 433us) server_negotiation.cc:407] Connection header received
0809 04:48:28.869467 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:28.869478 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:28.869780 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:28.870118 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:28.871288 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.872371 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.873675 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.874298 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.874507 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.877875 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:28.877882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:28.877901 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:28.877936 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:28.880272 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:28.881075 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:28.881081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:28.881087 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:28.881252 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:28.881741 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:28.881744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:28.881746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:28.882286 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:28.882424 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:28.883248 (+ 824us) server_negotiation.cc:299] Negotiation successful
0809 04:48:28.883577 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":138,"threads_started":1}
I20260809 04:48:28.884816 26008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.867476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:58463)
0809 04:48:28.868032 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:28.868130 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:28.869025 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:28.870172 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:28.870180 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:28.870624 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:28.871109 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:28.871123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.872548 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.872551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:28.873542 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:28.873553 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.874460 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.874467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:28.874668 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.875380 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:28.875397 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:28.877608 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:28.880419 (+ 2811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:28.880430 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:28.880447 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:28.880901 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:28.881375 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:28.881382 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:28.881386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:28.881610 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:28.882433 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:28.882441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:28.882833 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 04:48:28.883110 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 04:48:28.883549 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":182,"threads_started":1}
I20260809 04:48:28.885964 25884 ts_tablet_manager.cc:1397] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Time spent bootstrapping tablet: real 0.035s user 0.018s sys 0.015s
I20260809 04:48:28.888134 26004 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34927
I20260809 04:48:28.888423 26004 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:28.889103 26004 heartbeater.cc:511] Master 127.7.171.62:34927 requested a full tablet report, sending...
I20260809 04:48:28.888828 25884 raft_consensus.cc:348] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } }
I20260809 04:48:28.889264 25884 raft_consensus.cc:729] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262a6b80c3fc47bc8a752ed6c9a1b4cb, State: Initialized, Role: FOLLOWER
I20260809 04:48:28.889736 25884 consensus_queue.cc:260] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } }
I20260809 04:48:28.890704 25884 ts_tablet_manager.cc:1428] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 04:48:28.892555 25801 ts_manager.cc:194] Registered new tserver with Master: 262a6b80c3fc47bc8a752ed6c9a1b4cb (127.7.171.1:36525)
I20260809 04:48:28.896193 25801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:58463
I20260809 04:48:28.898797 26004 heartbeater.cc:503] Master 127.7.171.62:34927 was elected leader, sending a full tablet report...
Aug 09 04:48:28 dist-test-slave-1zqn krb5kdc[25167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250908, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.1@KRBTEST.COM
I20260809 04:48:28.907430 26012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.895596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36525 (local address 127.0.0.1:51686)
0809 04:48:28.896224 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:28.896271 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:28.896454 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:28.897177 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:28.897184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:28.897676 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:28.897814 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:28.897822 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.899138 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.899142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:28.900324 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:28.900337 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.900942 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.900947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:28.901084 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.901670 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:28.901691 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:28.903452 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:28.905549 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:28.905558 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:28.905564 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:28.905810 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:28.906155 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:28.906158 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:28.906160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:28.906253 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:28.906750 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:28.906756 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:28.906883 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 04:48:28.907037 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 04:48:28.907246 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":122,"threads_started":1}
I20260809 04:48:28.907430 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.895644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51686 (local address 127.7.171.1:36525)
0809 04:48:28.896516 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:28.896528 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:28.896555 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:48:28.896630 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:28.896635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:28.896839 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:28.896991 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:28.897961 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.898992 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.900450 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.900812 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.900900 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.903597 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:28.903607 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:28.903625 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:28.903648 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:28.905388 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:28.905931 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:28.905937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:28.905940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:28.906039 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:28.906345 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:28.906350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:28.906351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:28.906630 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:28.906773 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:28.907120 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 04:48:28.907283 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":98,"threads_started":1}
I20260809 04:48:28.908430 25932 tablet_service.cc:1514] Processing DeleteTablet for tablet cb4c06635b544f7c81cfcb477915e619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:08 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:51686
I20260809 04:48:28.909340 26014 tablet_replica.cc:330] stopping tablet replica
I20260809 04:48:28.909715 26014 raft_consensus.cc:2229] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:28.910023 26014 raft_consensus.cc:2258] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:28.911243 26014 ts_tablet_manager.cc:1905] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:28.916309 26014 ts_tablet_manager.cc:1918] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:28.916433 26014 log.cc:1196] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal/wals/cb4c06635b544f7c81cfcb477915e619
I20260809 04:48:28.916757 26014 ts_tablet_manager.cc:1939] T cb4c06635b544f7c81cfcb477915e619 P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Deleting consensus metadata
I20260809 04:48:28.919140 25784 catalog_manager.cc:4841] TS 262a6b80c3fc47bc8a752ed6c9a1b4cb (127.7.171.1:36525): tablet cb4c06635b544f7c81cfcb477915e619 (table test-table [id=39476632fd1343b9b39997f4b9b96cfe]) successfully deleted
W20260809 04:48:29.004470 26010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:29.004740 26010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:29.004786 26010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:29.008826 26010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:29.008889 26010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:29.008991 26010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:48:29.014295 26010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:45883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=41617
--tserver_master_addrs=127.7.171.62:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:29.015429 26010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:29.016691 26010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:29.027266 26022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:29.027266 26020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:29.027660 26010 server_base.cc:1034] running on GCE node
W20260809 04:48:29.027277 26019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:29.028332 26010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:29.029021 26010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:29.030220 26010 hybrid_clock.cc:648] HybridClock initialized: now 1786250909030199 us; error 35 us; skew 500 ppm
Aug 09 04:48:29 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250909, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:29.033288 26010 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:48:29.034235 26010 webserver.cc:460] Webserver started at http://127.7.171.2:41617/ using document root <none> and password file <none>
I20260809 04:48:29.034816 26010 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:29.034933 26010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:29.038931 26010 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:48:29.041301 26029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:29.042590 26010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:48:29.042732 26010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "5ea5833ff17d42f5b76c082fe8f99413"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "7c1a04306136ca6b4370495a992804dd"
server_key_iv: "a832d02b71676b05dc3cb4271c0c124c"
server_key_version: "encryptionkey@0"
I20260809 04:48:29.043377 26010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:29.062899 26010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:29.066315 26010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:29.066533 26010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:29.067150 26010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:29.068423 26038 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260809 04:48:29.072547 26038 ts_tablet_manager.cc:1852] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260809 04:48:29.073000 26038 ts_tablet_manager.cc:1905] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:29.077280 26038 ts_tablet_manager.cc:1918] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:29.077414 26038 log.cc:1196] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/cb4c06635b544f7c81cfcb477915e619
I20260809 04:48:29.077723 26038 ts_tablet_manager.cc:1939] T cb4c06635b544f7c81cfcb477915e619 P 5ea5833ff17d42f5b76c082fe8f99413: Deleting consensus metadata
I20260809 04:48:29.078099 26010 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260809 04:48:29.078200 26010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.000s
I20260809 04:48:29.078297 26010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:29.078341 26010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:29.088548 26010 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:45883
I20260809 04:48:29.088593 26151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:45883 every 8 connection(s)
I20260809 04:48:29.089542 26010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:48:29.093577 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 26010
I20260809 04:48:29.094238 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:45159
--local_ip_for_outbound_sockets=127.7.171.3
--tserver_master_addrs=127.7.171.62:34927
--webserver_port=36395
--webserver_interface=127.7.171.3
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:48:29 dist-test-slave-1zqn krb5kdc[25167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250909, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:48:29.105435 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.091583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:55153 (local address 127.7.171.62:34927)
0809 04:48:29.091727 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.091731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.092634 (+ 903us) server_negotiation.cc:407] Connection header received
0809 04:48:29.093730 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.093735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.093801 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.093908 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:29.095313 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.096330 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.097716 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.098005 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.098137 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.101080 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:29.101083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:29.101086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:29.101114 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:29.103050 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:29.103655 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:29.103660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:29.103662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:29.103750 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:29.104200 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:29.104205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:29.104207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:29.104406 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:29.104521 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:29.105083 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 04:48:29.105237 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:48:29.106312 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.091927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:55153)
0809 04:48:29.092466 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:29.092512 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:29.093431 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:29.094078 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:29.094085 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:29.094555 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:29.095154 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.095169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.096541 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.096544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.097566 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.097577 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.098185 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.098190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.098379 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.099087 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:29.099105 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:29.100871 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:29.103183 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:29.103190 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:29.103206 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:29.103536 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:29.103857 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:29.103862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:29.103864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:29.104030 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:29.104535 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:29.104545 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:29.104812 (+ 267us) client_negotiation.cc:769] Sending connection context
0809 04:48:29.105014 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 04:48:29.105394 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":168,"threads_started":1}
I20260809 04:48:29.107709 26152 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34927
I20260809 04:48:29.108119 26152 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:29.108706 26152 heartbeater.cc:511] Master 127.7.171.62:34927 requested a full tablet report, sending...
I20260809 04:48:29.109980 25801 ts_manager.cc:194] Registered new tserver with Master: 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883)
I20260809 04:48:29.110842 25801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:55153
W20260809 04:48:29.220371 26155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:29.220644 26155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:29.220690 26155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:29.224721 26155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:29.224779 26155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:29.224836 26155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:48:29.230156 26155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:45159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=36395
--tserver_master_addrs=127.7.171.62:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:29.231174 26155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:29.232230 26155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:29.242138 26160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:29.242137 26161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:29.242205 26163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:29.242533 26155 server_base.cc:1034] running on GCE node
I20260809 04:48:29.242892 26155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:29.243611 26155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:29.244817 26155 hybrid_clock.cc:648] HybridClock initialized: now 1786250909244793 us; error 38 us; skew 500 ppm
Aug 09 04:48:29 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250909, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:29.247923 26155 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:48:29.249012 26155 webserver.cc:460] Webserver started at http://127.7.171.3:36395/ using document root <none> and password file <none>
I20260809 04:48:29.249590 26155 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:29.249671 26155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:29.253533 26155 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:48:29.255654 26170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:29.257181 26155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:48:29.257326 26155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "1c2e8a79ed2b46eaac833bf2161a258e"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "a7112678d823f37f5f645fe3ab4a1a22"
server_key_iv: "c0199cc97dc89038ec70f7948b66d8b4"
server_key_version: "encryptionkey@0"
I20260809 04:48:29.257882 26155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:29.272681 26155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:29.276405 26155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:29.276629 26155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:29.277307 26155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:29.278232 26155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:29.278306 26155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:29.278380 26155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:29.278426 26155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:29.288770 26155 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:45159
I20260809 04:48:29.288795 26296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:45159 every 8 connection(s)
I20260809 04:48:29.289844 26155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:48:29.293129 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 26155
Aug 09 04:48:29 dist-test-slave-1zqn krb5kdc[25167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250909, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:48:29.302739 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.293848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54328 (local address 127.7.171.62:34927)
0809 04:48:29.294170 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.294175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.294439 (+ 264us) server_negotiation.cc:407] Connection header received
0809 04:48:29.294621 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.294624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.294715 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.294813 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:29.295705 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.296379 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.297237 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.297551 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.297661 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.299268 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:29.299272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:29.299275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:29.299298 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:29.301103 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:29.301530 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:29.301534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:29.301536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:29.301621 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:29.301889 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:29.301894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:29.301896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:29.302070 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:29.302285 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:29.302427 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 04:48:29.302552 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"spinlock_wait_cycles":24576,"thread_start_us":98,"threads_started":1}
I20260809 04:48:29.304383 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.291878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:36105 (local address 127.7.171.62:34927)
0809 04:48:29.292025 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.292028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.293260 (+ 1232us) server_negotiation.cc:407] Connection header received
0809 04:48:29.294443 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.294446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.294508 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.294609 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:29.295882 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.296546 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.297678 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.297890 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.298007 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.300418 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:29.300424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:29.300431 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:29.300454 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:29.301969 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:29.302506 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:29.302509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:29.302513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:29.302610 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:29.303050 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:29.303054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:29.303056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:29.303266 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:29.303365 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:29.304119 (+ 754us) server_negotiation.cc:299] Negotiation successful
0809 04:48:29.304240 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57,"spinlock_wait_cycles":6528}
I20260809 04:48:29.305150 26300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.292254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:36105)
0809 04:48:29.293060 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:29.293125 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:29.294196 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:29.294828 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:29.294835 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:29.295192 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:29.295738 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.295753 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.296680 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.296683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.297551 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.297560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.298049 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.298053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.298218 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.298773 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:29.298789 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:29.300261 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:29.302097 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:29.302102 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:29.302117 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:29.302373 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:29.302704 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:29.302707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:29.302709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:29.302895 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:29.303385 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:29.303391 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:29.303708 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 04:48:29.303927 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 04:48:29.304261 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":124,"threads_started":1}
I20260809 04:48:29.306241 26297 heartbeater.cc:347] Connected to a master server at 127.7.171.62:34927
I20260809 04:48:29.306506 26297 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:29.307009 26297 heartbeater.cc:511] Master 127.7.171.62:34927 requested a full tablet report, sending...
I20260809 04:48:29.308104 25801 ts_manager.cc:194] Registered new tserver with Master: 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159)
I20260809 04:48:29.308665 25801 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:36105
I20260809 04:48:29.317037 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:48:29.321388 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.317656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.7.171.62:34927)
0809 04:48:29.317798 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.317802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.318021 (+ 219us) server_negotiation.cc:407] Connection header received
0809 04:48:29.318172 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.318174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.318285 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.318382 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:48:29.318732 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.319268 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.320330 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.320571 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.320662 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.320929 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:48:29.321066 (+ 137us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 04:48:29.321285 (+ 219us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54338: token expired
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 04:48:29.321497 26007 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54338: token expired
W20260809 04:48:29.321803 25722 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.171.62:34927)
I20260809 04:48:29.329780 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.322177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.7.171.62:34927)
0809 04:48:29.322312 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.322316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.322330 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:48:29.322412 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.322414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.322474 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.322559 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:29.323228 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.323709 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.324564 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.324774 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.324880 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.326286 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:29.326290 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:29.326292 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:29.326313 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:29.327867 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:29.328461 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:29.328467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:29.328468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:29.328582 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:29.328868 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:29.328872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:29.328873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:29.329050 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:29.329220 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:29.329476 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 04:48:29.329612 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:48:29.332392 25801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:48:29.335731 25801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:48:29.343650 25932 tablet_service.cc:1467] Processing CreateTablet for tablet 04c3eb37159e4e43bc28845e9e2759ed (DEFAULT_TABLE table=test-table [id=fbca631114c34813bd5dd642a0e02c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:29.344494 25932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04c3eb37159e4e43bc28845e9e2759ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:29.347792 25884 tablet_bootstrap.cc:492] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Bootstrap starting.
I20260809 04:48:29.348178 26012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.342519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45159 (local address 127.0.0.1:41322)
0809 04:48:29.342653 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:29.342671 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:29.342763 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:29.343753 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:29.343756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:29.343777 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:29.343909 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.343917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.345376 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.345379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.346575 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.346585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.347654 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.347658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.347768 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.347825 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 04:48:29.347910 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 04:48:29.348006 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":39}
I20260809 04:48:29.348555 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.342680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.7.171.3:45159)
0809 04:48:29.343171 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.343187 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.343216 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:48:29.343303 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.343310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.343573 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.343752 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:29.344093 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.345248 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.346758 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.347524 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.347634 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.347876 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:29.347939 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 04:48:29.348397 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":167,"threads_started":1}
I20260809 04:48:29.348862 25884 tablet_bootstrap.cc:654] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:29.349051 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.343006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:45883 (local address 127.0.0.1:42320)
0809 04:48:29.343389 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:29.343403 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:29.343484 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:29.344292 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:29.344294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:29.344314 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:29.344422 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.344427 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.345847 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.345850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.347070 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.347082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.348621 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.348627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.348741 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.348769 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 04:48:29.348825 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 04:48:29.348894 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":28,"thread_start_us":101,"threads_started":1}
I20260809 04:48:29.349825 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.343364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.7.171.2:45883)
0809 04:48:29.343766 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.343778 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.343801 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:48:29.343878 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.343883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.344127 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.344576 (+ 449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:29.344770 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.345739 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.347215 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.348216 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.349342 (+ 1126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.349495 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:29.349564 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 04:48:29.349668 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":123,"threads_started":1}
I20260809 04:48:29.350407 25884 tablet_bootstrap.cc:492] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: No bootstrap required, opened a new log
I20260809 04:48:29.350533 25884 ts_tablet_manager.cc:1397] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:48:29.351004 25884 raft_consensus.cc:348] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:29.351135 25884 raft_consensus.cc:374] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:29.351176 25884 raft_consensus.cc:729] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262a6b80c3fc47bc8a752ed6c9a1b4cb, State: Initialized, Role: FOLLOWER
I20260809 04:48:29.351316 25884 consensus_queue.cc:260] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:29.351598 25884 ts_tablet_manager.cc:1428] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 04:48:29.351861 26084 tablet_service.cc:1467] Processing CreateTablet for tablet 04c3eb37159e4e43bc28845e9e2759ed (DEFAULT_TABLE table=test-table [id=fbca631114c34813bd5dd642a0e02c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:29.351905 26225 tablet_service.cc:1467] Processing CreateTablet for tablet 04c3eb37159e4e43bc28845e9e2759ed (DEFAULT_TABLE table=test-table [id=fbca631114c34813bd5dd642a0e02c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:29.352973 26084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04c3eb37159e4e43bc28845e9e2759ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:29.353063 26225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04c3eb37159e4e43bc28845e9e2759ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:29.357458 26038 tablet_bootstrap.cc:492] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413: Bootstrap starting.
I20260809 04:48:29.359844 26038 tablet_bootstrap.cc:654] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:29.360847 26038 log.cc:824] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:29.361455 26311 tablet_bootstrap.cc:492] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e: Bootstrap starting.
I20260809 04:48:29.363240 26038 tablet_bootstrap.cc:492] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413: No bootstrap required, opened a new log
I20260809 04:48:29.363464 26038 ts_tablet_manager.cc:1397] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 04:48:29.364190 26311 tablet_bootstrap.cc:654] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:29.365916 26311 log.cc:824] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e: Log is configured to *not* fsync() on all Append() calls
W20260809 04:48:29.366457 26006 tablet.cc:2369] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:48:29.367126 26038 raft_consensus.cc:348] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:29.367426 26038 raft_consensus.cc:374] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:29.367491 26038 raft_consensus.cc:729] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ea5833ff17d42f5b76c082fe8f99413, State: Initialized, Role: FOLLOWER
I20260809 04:48:29.368027 26038 consensus_queue.cc:260] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:29.369058 26311 tablet_bootstrap.cc:492] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e: No bootstrap required, opened a new log
I20260809 04:48:29.369292 26311 ts_tablet_manager.cc:1397] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20260809 04:48:29.369598 26038 ts_tablet_manager.cc:1428] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 04:48:29.369977 26152 heartbeater.cc:503] Master 127.7.171.62:34927 was elected leader, sending a full tablet report...
I20260809 04:48:29.373404 26311 raft_consensus.cc:348] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:29.373668 26311 raft_consensus.cc:374] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:29.373735 26311 raft_consensus.cc:729] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c2e8a79ed2b46eaac833bf2161a258e, State: Initialized, Role: FOLLOWER
I20260809 04:48:29.374202 26311 consensus_queue.cc:260] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:29.374881 26297 heartbeater.cc:503] Master 127.7.171.62:34927 was elected leader, sending a full tablet report...
I20260809 04:48:29.375226 26311 ts_tablet_manager.cc:1428] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:48:29.419322 26011 raft_consensus.cc:482] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:29.419565 26011 raft_consensus.cc:504] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:29.420830 26011 leader_election.cc:288] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159), 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883)
I20260809 04:48:29.424793 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.421290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:35645 (local address 127.7.171.2:45883)
0809 04:48:29.421410 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.421418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.421599 (+ 181us) server_negotiation.cc:407] Connection header received
0809 04:48:29.421743 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.421746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.421790 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.421881 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:29.422203 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.422695 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.423736 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.424186 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.424334 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.424397 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:29.424595 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 04:48:29.424666 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:48:29.424827 26317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.421186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:45883 (local address 127.7.171.1:35645)
0809 04:48:29.421507 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:29.421520 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:29.421609 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:29.421894 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:29.421897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:29.421913 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:29.422076 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.422082 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.422815 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.422820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.423604 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.423613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.424340 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.424344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.424435 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.424485 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 04:48:29.424578 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 04:48:29.424656 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":67,"threads_started":1}
I20260809 04:48:29.424983 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.421192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:48839 (local address 127.7.171.3:45159)
0809 04:48:29.421335 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.421339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.421471 (+ 132us) server_negotiation.cc:407] Connection header received
0809 04:48:29.421697 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.421700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.421746 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.421833 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:29.422123 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.422631 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.423829 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.424366 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.424494 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.424555 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:29.424757 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 04:48:29.424823 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:48:29.425081 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.421035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45159 (local address 127.7.171.1:48839)
0809 04:48:29.421383 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:29.421398 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:29.421532 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:29.421851 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:29.421854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:29.421871 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:29.422008 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.422013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.422744 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.422749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.423709 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.423719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.424495 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.424498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.424608 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.424637 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:48:29.424738 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 04:48:29.424814 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":100,"threads_started":1}
I20260809 04:48:29.425691 26104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c3eb37159e4e43bc28845e9e2759ed" candidate_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ea5833ff17d42f5b76c082fe8f99413" is_pre_election: true
I20260809 04:48:29.425748 26248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c3eb37159e4e43bc28845e9e2759ed" candidate_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" is_pre_election: true
I20260809 04:48:29.426028 26104 raft_consensus.cc:2454] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262a6b80c3fc47bc8a752ed6c9a1b4cb in term 0.
I20260809 04:48:29.426157 26248 raft_consensus.cc:2454] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262a6b80c3fc47bc8a752ed6c9a1b4cb in term 0.
I20260809 04:48:29.426566 25881 leader_election.cc:302] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262a6b80c3fc47bc8a752ed6c9a1b4cb, 5ea5833ff17d42f5b76c082fe8f99413; no voters:
I20260809 04:48:29.426887 26011 raft_consensus.cc:2781] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:48:29.426980 26011 raft_consensus.cc:482] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:48:29.427012 26011 raft_consensus.cc:3037] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:29.428222 26011 raft_consensus.cc:504] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:29.428668 26011 leader_election.cc:288] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 1 election: Requested vote from peers 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159), 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883)
I20260809 04:48:29.429162 26248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c3eb37159e4e43bc28845e9e2759ed" candidate_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c2e8a79ed2b46eaac833bf2161a258e"
I20260809 04:48:29.429195 26104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04c3eb37159e4e43bc28845e9e2759ed" candidate_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ea5833ff17d42f5b76c082fe8f99413"
I20260809 04:48:29.429270 26248 raft_consensus.cc:3037] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:29.429353 26104 raft_consensus.cc:3037] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:29.430400 26248 raft_consensus.cc:2454] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262a6b80c3fc47bc8a752ed6c9a1b4cb in term 1.
I20260809 04:48:29.430541 26104 raft_consensus.cc:2454] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262a6b80c3fc47bc8a752ed6c9a1b4cb in term 1.
I20260809 04:48:29.430779 25879 leader_election.cc:302] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c2e8a79ed2b46eaac833bf2161a258e, 262a6b80c3fc47bc8a752ed6c9a1b4cb; no voters:
I20260809 04:48:29.430945 26011 raft_consensus.cc:2781] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:29.431173 26011 raft_consensus.cc:686] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 LEADER]: Becoming Leader. State: Replica: 262a6b80c3fc47bc8a752ed6c9a1b4cb, State: Running, Role: LEADER
I20260809 04:48:29.431452 26011 consensus_queue.cc:237] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:29.434566 25801 catalog_manager.cc:5495] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 262a6b80c3fc47bc8a752ed6c9a1b4cb (127.7.171.1). New cstate: current_term: 1 leader_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } health_report { overall_health: HEALTHY } } }
I20260809 04:48:29.460292 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.456401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51694 (local address 127.7.171.1:36525)
0809 04:48:29.456697 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.456704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.456718 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:48:29.456797 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.456799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.456856 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.456993 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:48:29.457277 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.457795 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.458787 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.459019 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.459117 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.459345 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:48:29.459812 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:48:29.459899 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:29.460097 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 04:48:29.460151 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":129,"threads_started":1}
I20260809 04:48:29.464443 26248 raft_consensus.cc:1261] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e [term 1 FOLLOWER]: Refusing update from remote peer 262a6b80c3fc47bc8a752ed6c9a1b4cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:29.464459 26104 raft_consensus.cc:1261] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413 [term 1 FOLLOWER]: Refusing update from remote peer 262a6b80c3fc47bc8a752ed6c9a1b4cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:29.465193 26318 consensus_queue.cc:1035] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:29.465365 26011 consensus_queue.cc:1035] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:29.472463 26325 mvcc.cc:205] Tried to move back new op lower bound from 7316483725159804928 to 7316483725034602496. Current Snapshot: MvccSnapshot[applied={T|T < 7316483725159804928}]
I20260809 04:48:29.481320 25801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:29.481520 25801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:29.484097 25801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Sending DeleteTablet for 3 replicas of tablet 04c3eb37159e4e43bc28845e9e2759ed
I20260809 04:48:29.484839 26084 tablet_service.cc:1514] Processing DeleteTablet for tablet 04c3eb37159e4e43bc28845e9e2759ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:29 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:42320
I20260809 04:48:29.484838 26225 tablet_service.cc:1514] Processing DeleteTablet for tablet 04c3eb37159e4e43bc28845e9e2759ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:29 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:41322
I20260809 04:48:29.485569 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 25840
I20260809 04:48:29.485591 26333 tablet_replica.cc:330] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413: stopping tablet replica
I20260809 04:48:29.485952 26333 raft_consensus.cc:2229] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:29.486346 26333 raft_consensus.cc:2258] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:29.486689 26332 tablet_replica.cc:330] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e: stopping tablet replica
I20260809 04:48:29.487005 26332 raft_consensus.cc:2229] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:29.487322 26332 raft_consensus.cc:2258] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:29.488709 26333 ts_tablet_manager.cc:1905] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:29.488801 26332 ts_tablet_manager.cc:1905] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:29.491433 26333 ts_tablet_manager.cc:1918] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:29.491428 26332 ts_tablet_manager.cc:1918] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:29.491518 26333 log.cc:1196] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/04c3eb37159e4e43bc28845e9e2759ed
I20260809 04:48:29.491528 26332 log.cc:1196] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/04c3eb37159e4e43bc28845e9e2759ed
I20260809 04:48:29.491827 26333 ts_tablet_manager.cc:1939] T 04c3eb37159e4e43bc28845e9e2759ed P 5ea5833ff17d42f5b76c082fe8f99413: Deleting consensus metadata
I20260809 04:48:29.491868 26332 ts_tablet_manager.cc:1939] T 04c3eb37159e4e43bc28845e9e2759ed P 1c2e8a79ed2b46eaac833bf2161a258e: Deleting consensus metadata
I20260809 04:48:29.493131 25785 catalog_manager.cc:4841] TS 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883): tablet 04c3eb37159e4e43bc28845e9e2759ed (table test-table [id=fbca631114c34813bd5dd642a0e02c41]) successfully deleted
I20260809 04:48:29.493131 25783 catalog_manager.cc:4841] TS 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159): tablet 04c3eb37159e4e43bc28845e9e2759ed (table test-table [id=fbca631114c34813bd5dd642a0e02c41]) successfully deleted
W20260809 04:48:29.496599 25784 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:36525 (error 108)
I20260809 04:48:29.496848 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 26010
I20260809 04:48:29.498039 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.497630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36525 (local address 127.0.0.1:51696)
0809 04:48:29.497830 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:29.497926 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:36525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260809 04:48:29.498226 25784 catalog_manager.cc:4584] TS 262a6b80c3fc47bc8a752ed6c9a1b4cb (127.7.171.1:36525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 04c3eb37159e4e43bc28845e9e2759ed: Network error: Client connection negotiation failed: client connection to 127.7.171.1:36525: connect: Connection refused (error 111)
I20260809 04:48:29.504964 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 26155
I20260809 04:48:29.513370 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 25763
I20260809 04:48:49.522115 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34927
--webserver_interface=127.7.171.62
--webserver_port=34499
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:34927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:49.649907 26335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:49.650177 26335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:49.650216 26335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:49.654265 26335 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 04:48:49.654323 26335 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 04:48:49.654340 26335 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 04:48:49.654353 26335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:49.654371 26335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:49.654388 26335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:49.654428 26335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:49.654465 26335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:49.659983 26335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:34927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:34927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=34499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:49.661069 26335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:49.662137 26335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:49.671279 26340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:49.671420 26335 server_base.cc:1034] running on GCE node
W20260809 04:48:49.671300 26343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:49.671280 26341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:49.672544 26335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:49.673700 26335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:49.674870 26335 hybrid_clock.cc:648] HybridClock initialized: now 1786250929674840 us; error 60 us; skew 500 ppm
Aug 09 04:48:49 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250929, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:49.685423 26335 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:48:49.686450 26335 webserver.cc:460] Webserver started at http://127.7.171.62:34499/ using document root <none> and password file <none>
I20260809 04:48:49.687039 26335 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:49.687114 26335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:49.691171 26335 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:48:49.693380 26353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:49.694628 26335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:48:49.694761 26335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "56576a93975b4e8196f83720d69136bc"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "2b2fb5250c57a13c4cf632f680ad91b7"
server_key_iv: "5a6b35a2691cf2b3f523bd3478af6aad"
server_key_version: "encryptionkey@0"
I20260809 04:48:49.695304 26335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:49.710292 26335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:49.714128 26335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:49.714365 26335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:49.722563 26335 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:34927
I20260809 04:48:49.723029 26407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:34927 every 8 connection(s)
I20260809 04:48:49.723843 26335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:49.728606 26408 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:48:49.730370 26408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Bootstrap starting.
I20260809 04:48:49.731386 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 26335
I20260809 04:48:49.731877 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:36525
--local_ip_for_outbound_sockets=127.7.171.1
--tserver_master_addrs=127.7.171.62:34927
--webserver_port=40067
--webserver_interface=127.7.171.1
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:48:49.749861 26408 log.cc:824] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:49.761794 26408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:48:49.762220 26408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Bootstrap complete.
I20260809 04:48:49.766578 26408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:49.767022 26408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56576a93975b4e8196f83720d69136bc, State: Initialized, Role: FOLLOWER
I20260809 04:48:49.767587 26408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:49.767740 26408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:49.767833 26408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:49.767954 26408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 2 FOLLOWER]: Advancing to term 3
I20260809 04:48:49.769990 26408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:49.770340 26408 leader_election.cc:302] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56576a93975b4e8196f83720d69136bc; no voters:
I20260809 04:48:49.770742 26408 leader_election.cc:288] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 04:48:49.770830 26415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 3 FOLLOWER]: Leader election won for term 3
I20260809 04:48:49.771024 26415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [term 3 LEADER]: Becoming Leader. State: Replica: 56576a93975b4e8196f83720d69136bc, State: Running, Role: LEADER
I20260809 04:48:49.771325 26415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } }
I20260809 04:48:49.772044 26408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:49.772310 26417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56576a93975b4e8196f83720d69136bc. Latest consensus state: current_term: 3 leader_uuid: "56576a93975b4e8196f83720d69136bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } } }
I20260809 04:48:49.772433 26417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:49.772831 26419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:49.772845 26416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "56576a93975b4e8196f83720d69136bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56576a93975b4e8196f83720d69136bc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 34927 } } }
I20260809 04:48:49.772982 26416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:49.778122 26419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:49.778824 26419 catalog_manager.cc:1259] Loaded cluster ID: d186f7eb702a42a88162c825243bc9a6
I20260809 04:48:49.778892 26419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:49.781603 26419 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:49.782356 26419 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Loaded TSK: 0
I20260809 04:48:49.782442 26419 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 56576a93975b4e8196f83720d69136bc: Loaded TSK: 1
F20260809 04:48:49.783221 26419 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe7701899ad google::LogMessage::Fail() at ??:0
@ 0x7fe77018c88a google::LogMessage::SendToLog() at ??:0
@ 0x7fe7701894de google::LogMessage::Flush() at ??:0
@ 0x7fe77018aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe772e4d18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe772e45f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe77aaee7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe77aac899c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe77aac9178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fe77ab0adf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fe77ab2982b std::function<>::operator()() at ??:0
@ 0x7fe77aac8b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe77aac9f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe77aac6579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe77ab0a640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe7723db3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe7723dbc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe7723dd3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe7723cc512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe773a106db start_thread at ??:0
@ 0x7fe76ec8761f clone at ??:0
W20260809 04:48:49.872694 26411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:49.872943 26411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:49.872988 26411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:49.877465 26411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:49.877523 26411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:49.877586 26411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:48:49.883647 26411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:36525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=40067
--tserver_master_addrs=127.7.171.62:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:49.884825 26411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:49.885820 26411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:49.896039 26458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:49.896037 26459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:49.896323 26461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:49.896646 26411 server_base.cc:1034] running on GCE node
I20260809 04:48:49.897118 26411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:49.897919 26411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:49.899142 26411 hybrid_clock.cc:648] HybridClock initialized: now 1786250929899115 us; error 45 us; skew 500 ppm
Aug 09 04:48:49 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250929, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:49.902722 26411 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:48:49.903834 26411 webserver.cc:460] Webserver started at http://127.7.171.1:40067/ using document root <none> and password file <none>
I20260809 04:48:49.904590 26411 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:49.904690 26411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:49.909955 26411 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:48:49.913105 26470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:49.914772 26411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:48:49.914917 26411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "a62e712863fb948205cd886e5d4ad92a"
server_key_iv: "c27635c158a09ffb569072bc24930a4e"
server_key_version: "encryptionkey@0"
I20260809 04:48:49.915752 26411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:49.938212 26411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:49.942736 26411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:49.942993 26411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:49.943890 26411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:49.945513 26478 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 04:48:49.949697 26411 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 04:48:49.949784 26411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260809 04:48:49.949862 26411 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 04:48:49.952396 26411 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 04:48:49.952471 26411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260809 04:48:49.952749 26478 tablet_bootstrap.cc:492] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Bootstrap starting.
I20260809 04:48:49.966970 26411 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:36525
I20260809 04:48:49.968348 26597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:36525 every 8 connection(s)
I20260809 04:48:49.968464 26411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:48:49.970887 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 26411
I20260809 04:48:49.971395 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:45883
--local_ip_for_outbound_sockets=127.7.171.2
--tserver_master_addrs=127.7.171.62:34927
--webserver_port=41617
--webserver_interface=127.7.171.2
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:48:49.974339 26478 log.cc:824] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:49.984436 26478 tablet_bootstrap.cc:492] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:48:49.984963 26478 tablet_bootstrap.cc:492] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Bootstrap complete.
I20260809 04:48:49.986775 26478 ts_tablet_manager.cc:1397] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Time spent bootstrapping tablet: real 0.034s user 0.018s sys 0.013s
I20260809 04:48:49.990314 26478 raft_consensus.cc:348] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:49.990895 26478 raft_consensus.cc:729] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262a6b80c3fc47bc8a752ed6c9a1b4cb, State: Initialized, Role: FOLLOWER
I20260809 04:48:49.991456 26478 consensus_queue.cc:260] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:49.992579 26478 ts_tablet_manager.cc:1428] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260809 04:48:50.060639 26600 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:48:49.971265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:44485)
0809 04:48:49.971905 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:49.971970 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:49.973173 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:50.059680 (+ 86507us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":170,"threads_started":1}
W20260809 04:48:50.061408 26598 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:34927 (0 consecutive failures): Network error: Failed to ping master at 127.7.171.62:34927: Client connection negotiation failed: client connection to 127.7.171.62:34927: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 04:48:50.062247 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:50.061925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:40815)
0809 04:48:50.062106 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:50.062178 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 04:48:50.062956 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:50.062750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:33005)
0809 04:48:50.062860 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:50.062910 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 04:48:50.063159 26598 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:48:50.098515 26601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:50.098834 26601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:50.098932 26601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:50.103123 26601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:50.103207 26601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:50.103307 26601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:48:50.108671 26601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:45883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=41617
--tserver_master_addrs=127.7.171.62:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:50.109754 26601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:50.110796 26601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:50.121104 26610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:50.121142 26613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:50.120950 26611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:50.121482 26601 server_base.cc:1034] running on GCE node
I20260809 04:48:50.121933 26601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:50.122673 26601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:50.123888 26601 hybrid_clock.cc:648] HybridClock initialized: now 1786250930123866 us; error 38 us; skew 500 ppm
Aug 09 04:48:50 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250930, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:50.127219 26601 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:48:50.128520 26601 webserver.cc:460] Webserver started at http://127.7.171.2:41617/ using document root <none> and password file <none>
I20260809 04:48:50.129331 26601 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:50.129402 26601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:50.133821 26601 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:48:50.135946 26620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:50.137409 26601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:48:50.137569 26601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "5ea5833ff17d42f5b76c082fe8f99413"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "7c1a04306136ca6b4370495a992804dd"
server_key_iv: "a832d02b71676b05dc3cb4271c0c124c"
server_key_version: "encryptionkey@0"
I20260809 04:48:50.138250 26601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:50.162528 26601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:50.166247 26601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:50.166484 26601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:50.167214 26601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:50.168169 26601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:50.168305 26601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:50.168387 26601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:50.168428 26601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:50.178858 26601 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:45883
I20260809 04:48:50.178889 26747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:45883 every 8 connection(s)
I20260809 04:48:50.179934 26601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:48:50.182106 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 26601
I20260809 04:48:50.182574 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:45159
--local_ip_for_outbound_sockets=127.7.171.3
--tserver_master_addrs=127.7.171.62:34927
--webserver_port=36395
--webserver_interface=127.7.171.3
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:41571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:48:50.184248 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:50.182329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:41851)
0809 04:48:50.183211 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:50.183357 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":155,"threads_started":1}
W20260809 04:48:50.184862 26748 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:34927 (0 consecutive failures): Network error: Failed to ping master at 127.7.171.62:34927: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
I20260809 04:48:50.185554 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:50.185290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:50551)
0809 04:48:50.185420 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:50.185487 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 04:48:50.186383 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:50.186072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:35501)
0809 04:48:50.186220 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:50.186299 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 04:48:50.186661 26748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:48:50.303880 26751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:50.304214 26751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:50.304265 26751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:50.308346 26751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:50.308409 26751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:50.308491 26751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:48:50.313959 26751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:41571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:45159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=36395
--tserver_master_addrs=127.7.171.62:34927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:48:50.315079 26751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:50.316164 26751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:50.326360 26759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:50.326521 26751 server_base.cc:1034] running on GCE node
W20260809 04:48:50.326368 26757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:50.326388 26756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:50.327261 26751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:50.327901 26751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:50.329118 26751 hybrid_clock.cc:648] HybridClock initialized: now 1786250930329116 us; error 58 us; skew 500 ppm
Aug 09 04:48:50 dist-test-slave-1zqn krb5kdc[25167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250930, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:50.332163 26751 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:48:50.333102 26751 webserver.cc:460] Webserver started at http://127.7.171.3:36395/ using document root <none> and password file <none>
I20260809 04:48:50.333752 26751 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:50.333856 26751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:50.338243 26751 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:48:50.340688 26766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:50.342088 26751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:48:50.342229 26751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "1c2e8a79ed2b46eaac833bf2161a258e"
format_stamp: "Formatted at 2026-08-09 04:48:07 on dist-test-slave-1zqn"
server_key: "a7112678d823f37f5f645fe3ab4a1a22"
server_key_iv: "c0199cc97dc89038ec70f7948b66d8b4"
server_key_version: "encryptionkey@0"
I20260809 04:48:50.342940 26751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:50.365756 26751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:50.369040 26751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:50.369275 26751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:50.370019 26751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:50.370949 26751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:50.371029 26751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:50.371109 26751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:50.371168 26751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:50.381102 26751 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:45159
I20260809 04:48:50.381140 26891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:45159 every 8 connection(s)
I20260809 04:48:50.382184 26751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:48:50.386060 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:50.384460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:40657)
0809 04:48:50.385184 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:50.385299 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":122,"threads_started":1}
W20260809 04:48:50.386515 26892 heartbeater.cc:644] Failed to heartbeat to 127.7.171.62:34927 (0 consecutive failures): Network error: Failed to ping master at 127.7.171.62:34927: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
I20260809 04:48:50.387195 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:50.386947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:34127)
0809 04:48:50.387076 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:50.387132 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 04:48:50.387825 26895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:50.387600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:51227)
0809 04:48:50.387715 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:50.387762 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 04:48:50.388037 26892 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 04:48:50.391455 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 26751
I20260809 04:48:51.064345 26897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:51.063858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:55411)
0809 04:48:51.064169 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:51.064222 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":113,"threads_started":1}
I20260809 04:48:51.187723 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:51.187267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:45051)
0809 04:48:51.187559 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:51.187619 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":73,"threads_started":1}
I20260809 04:48:51.389240 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:51.388667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:52521)
0809 04:48:51.389082 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:51.389140 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":77,"threads_started":1}
I20260809 04:48:51.853549 26902 raft_consensus.cc:482] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:51.853777 26902 raft_consensus.cc:504] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:51.855175 26902 leader_election.cc:288] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159), 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883)
Aug 09 04:48:51 dist-test-slave-1zqn krb5kdc[25167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250929, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.2@KRBTEST.COM
Aug 09 04:48:51 dist-test-slave-1zqn krb5kdc[25167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250929, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.3@KRBTEST.COM
I20260809 04:48:51.869346 26903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:51.855377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:45883 (local address 127.7.171.1:44703)
0809 04:48:51.855751 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:51.855765 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:51.855866 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:51.857525 (+ 1659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:51.857531 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:51.858131 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:51.858618 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:51.858644 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:51.860262 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:51.860266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:51.861098 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:51.861107 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:51.861846 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:51.861850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:51.862023 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:51.862704 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:51.862721 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:51.864335 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:51.866873 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:51.866879 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:51.866893 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:51.867146 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:51.867507 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:51.867510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:51.867512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:51.867655 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:51.868375 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:51.868381 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:51.868632 (+ 251us) client_negotiation.cc:769] Sending connection context
0809 04:48:51.868816 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 04:48:51.869152 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":23,"spinlock_wait_cycles":38528,"thread_start_us":73,"threads_started":1}
I20260809 04:48:51.869481 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:51.855532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:39503 (local address 127.7.171.3:45159)
0809 04:48:51.856038 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:51.856103 (+ 65us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:51.856149 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:48:51.856836 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:51.856841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:51.857162 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:51.857479 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:51.859534 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:51.860555 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:51.861353 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:51.861838 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:51.862019 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:51.864855 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:51.864859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:51.864870 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:51.864907 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:51.866903 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:51.867436 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:51.867447 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:51.867451 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:51.867570 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:51.867941 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:51.867944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:51.867945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:51.868396 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:51.868546 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:51.868958 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 04:48:51.869253 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":115,"threads_started":1}
I20260809 04:48:51.869783 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:51.855364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:45159 (local address 127.7.171.1:39503)
0809 04:48:51.855698 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:51.855715 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:51.855834 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:51.857525 (+ 1691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:51.857531 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:51.858131 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:51.858619 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:51.858647 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:51.860683 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:51.860688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:51.861222 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:51.861230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:51.861967 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:51.861970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:51.862064 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:51.862759 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:51.862772 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:51.864646 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:51.867075 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:51.867079 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:51.867081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:51.867306 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:51.867700 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:51.867710 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:51.867712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:51.867798 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:51.868528 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:51.868531 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:51.868655 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 04:48:51.868815 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 04:48:51.869256 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":187,"spinlock_wait_cycles":768,"thread_start_us":111,"threads_started":1}
I20260809 04:48:51.869396 26906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:51.855614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:44703 (local address 127.7.171.2:45883)
0809 04:48:51.856106 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:51.856125 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:51.856156 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:48:51.856816 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:51.856822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:51.857235 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:51.857517 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:51.858883 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:51.860128 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:51.861224 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:51.861710 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:51.861891 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:51.864596 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:51.864601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:51.864614 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:51.864651 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:51.866702 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:51.867270 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:51.867276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:51.867282 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:51.867402 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:51.867749 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:51.867754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:51.867756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:51.868248 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:51.868389 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:51.868949 (+ 560us) server_negotiation.cc:299] Negotiation successful
0809 04:48:51.869223 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":64,"threads_started":1}
W20260809 04:48:51.872233 26474 leader_election.cc:341] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883): Not found: Tablet not found: 04c3eb37159e4e43bc28845e9e2759ed
W20260809 04:48:51.872918 26472 leader_election.cc:341] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159): Not found: Tablet not found: 04c3eb37159e4e43bc28845e9e2759ed
I20260809 04:48:51.873031 26472 leader_election.cc:302] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262a6b80c3fc47bc8a752ed6c9a1b4cb; no voters: 1c2e8a79ed2b46eaac833bf2161a258e, 5ea5833ff17d42f5b76c082fe8f99413
I20260809 04:48:51.873287 26902 raft_consensus.cc:2726] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:48:52.065351 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:52.065075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:52005)
0809 04:48:52.065207 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:52.065260 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 04:48:52.189033 26907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:52.188528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:43547)
0809 04:48:52.188843 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:52.188912 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":97,"threads_started":1}
I20260809 04:48:52.390380 26909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:52.389967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:44867)
0809 04:48:52.390232 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:52.390288 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":86,"threads_started":1}
I20260809 04:48:53.066833 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:53.066309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:38415)
0809 04:48:53.066630 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:53.066703 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":100,"threads_started":1}
I20260809 04:48:53.190452 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:53.189997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:55773)
0809 04:48:53.190282 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:53.190350 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":88,"threads_started":1}
I20260809 04:48:53.391772 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:53.391296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:33803)
0809 04:48:53.391594 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:53.391657 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":101,"threads_started":1}
I20260809 04:48:53.946326 26913 raft_consensus.cc:482] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:53.946542 26913 raft_consensus.cc:504] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:53.947098 26913 leader_election.cc:288] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159), 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883)
W20260809 04:48:53.948118 26474 leader_election.cc:341] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883): Not found: Tablet not found: 04c3eb37159e4e43bc28845e9e2759ed
W20260809 04:48:53.948289 26472 leader_election.cc:341] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159): Not found: Tablet not found: 04c3eb37159e4e43bc28845e9e2759ed
I20260809 04:48:53.948379 26472 leader_election.cc:302] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262a6b80c3fc47bc8a752ed6c9a1b4cb; no voters: 1c2e8a79ed2b46eaac833bf2161a258e, 5ea5833ff17d42f5b76c082fe8f99413
I20260809 04:48:53.948539 26913 raft_consensus.cc:2726] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:48:54.068560 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:54.067806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:51933)
0809 04:48:54.068325 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:54.068417 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":160,"threads_started":1}
I20260809 04:48:54.191795 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:54.191362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:36429)
0809 04:48:54.191625 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:54.191680 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":88,"threads_started":1}
I20260809 04:48:54.392939 26916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:54.392517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:35527)
0809 04:48:54.392791 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:54.392850 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
I20260809 04:48:55.070273 26917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:55.069735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:38445)
0809 04:48:55.070070 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:55.070162 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":86,"threads_started":1}
I20260809 04:48:55.193393 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:55.192855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:37077)
0809 04:48:55.193220 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:55.193292 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":184,"threads_started":1}
I20260809 04:48:55.394236 26919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:55.393716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:44005)
0809 04:48:55.394053 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:55.394135 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":86,"threads_started":1}
I20260809 04:48:55.829337 26920 raft_consensus.cc:482] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:55.829483 26920 raft_consensus.cc:504] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:55.829993 26920 leader_election.cc:288] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159), 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883)
W20260809 04:48:55.830951 26474 leader_election.cc:341] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883): Not found: Tablet not found: 04c3eb37159e4e43bc28845e9e2759ed
W20260809 04:48:55.831043 26472 leader_election.cc:341] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159): Not found: Tablet not found: 04c3eb37159e4e43bc28845e9e2759ed
I20260809 04:48:55.831089 26474 leader_election.cc:302] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262a6b80c3fc47bc8a752ed6c9a1b4cb; no voters: 1c2e8a79ed2b46eaac833bf2161a258e, 5ea5833ff17d42f5b76c082fe8f99413
I20260809 04:48:55.831271 26920 raft_consensus.cc:2726] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:48:56.071762 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:56.071197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:35473)
0809 04:48:56.071535 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:56.071613 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":88,"threads_started":1}
I20260809 04:48:56.194881 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:56.194400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:56387)
0809 04:48:56.194695 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:56.194760 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":101,"threads_started":1}
I20260809 04:48:56.395610 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:56.395167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:40977)
0809 04:48:56.395449 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:56.395514 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":83,"threads_started":1}
I20260809 04:48:57.073096 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:57.072678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:53459)
0809 04:48:57.072931 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:57.072997 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260809 04:48:57.196385 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:57.195863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:52039)
0809 04:48:57.196218 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:57.196284 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":98,"threads_started":1}
I20260809 04:48:57.397150 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:57.396513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:51127)
0809 04:48:57.396919 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:57.397009 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":77,"threads_started":1}
I20260809 04:48:58.074509 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:58.073992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:38095)
0809 04:48:58.074308 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:58.074397 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":84,"threads_started":1}
I20260809 04:48:58.197950 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:58.197275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:46627)
0809 04:48:58.197713 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:58.197812 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":85,"threads_started":1}
I20260809 04:48:58.398470 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:58.398055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:44859)
0809 04:48:58.398316 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:58.398372 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":92,"threads_started":1}
I20260809 04:48:58.523460 26931 raft_consensus.cc:482] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:58.523628 26931 raft_consensus.cc:504] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:48:58.524021 26931 leader_election.cc:288] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159), 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883)
W20260809 04:48:58.524837 26474 leader_election.cc:341] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883): Not found: Tablet not found: 04c3eb37159e4e43bc28845e9e2759ed
W20260809 04:48:58.524955 26472 leader_election.cc:341] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159): Not found: Tablet not found: 04c3eb37159e4e43bc28845e9e2759ed
I20260809 04:48:58.525175 26472 leader_election.cc:302] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262a6b80c3fc47bc8a752ed6c9a1b4cb; no voters: 1c2e8a79ed2b46eaac833bf2161a258e, 5ea5833ff17d42f5b76c082fe8f99413
I20260809 04:48:58.525355 26931 raft_consensus.cc:2726] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:48:59.075865 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:59.075422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:43243)
0809 04:48:59.075697 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:59.075759 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":80,"threads_started":1}
I20260809 04:48:59.199328 26933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:59.198868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:33397)
0809 04:48:59.199162 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:59.199227 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":91,"threads_started":1}
I20260809 04:48:59.399889 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:59.399396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:49989)
0809 04:48:59.399720 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:59.399789 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":100,"threads_started":1}
I20260809 04:49:00.077287 26935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:00.076795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:34799)
0809 04:49:00.077106 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:00.077166 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":80,"threads_started":1}
I20260809 04:49:00.200699 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:00.200242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:51523)
0809 04:49:00.200531 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:00.200595 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":83,"threads_started":1}
I20260809 04:49:00.401329 26937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:00.400822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:46965)
0809 04:49:00.401145 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:00.401220 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":84,"threads_started":1}
I20260809 04:49:01.078725 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:01.078199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:57193)
0809 04:49:01.078539 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:01.078609 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":85,"threads_started":1}
I20260809 04:49:01.202061 26939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:01.201584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:33953)
0809 04:49:01.201890 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:01.201950 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":115,"threads_started":1}
I20260809 04:49:01.402647 26940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:01.402197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:33857)
0809 04:49:01.402484 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:01.402546 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":99,"threads_started":1}
I20260809 04:49:02.080211 26941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:02.079654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:60019)
0809 04:49:02.079953 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:02.080030 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":82,"threads_started":1}
I20260809 04:49:02.203418 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:02.202924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:50049)
0809 04:49:02.203235 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:02.203300 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":96,"threads_started":1}
I20260809 04:49:02.404119 26944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:02.403569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:34115)
0809 04:49:02.403847 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:02.403907 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":83,"threads_started":1}
I20260809 04:49:03.081597 26945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:03.081158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:45565)
0809 04:49:03.081427 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:03.081497 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":82,"threads_started":1}
I20260809 04:49:03.204737 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:03.204263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:39123)
0809 04:49:03.204572 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:03.204636 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":87,"threads_started":1}
I20260809 04:49:03.405411 26947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:03.404936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:58053)
0809 04:49:03.405252 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:03.405311 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":146,"threads_started":1}
I20260809 04:49:04.083078 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:04.082587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:55713)
0809 04:49:04.082897 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:04.082965 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":87,"threads_started":1}
I20260809 04:49:04.206077 26949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:04.205622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:39043)
0809 04:49:04.205892 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:04.205967 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":83,"threads_started":1}
I20260809 04:49:04.406718 26950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:04.406313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:44591)
0809 04:49:04.406564 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:04.406618 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":79,"threads_started":1}
I20260809 04:49:04.640561 26951 raft_consensus.cc:482] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:49:04.640702 26951 raft_consensus.cc:504] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c2e8a79ed2b46eaac833bf2161a258e" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 45159 } } peers { permanent_uuid: "5ea5833ff17d42f5b76c082fe8f99413" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 45883 } } peers { permanent_uuid: "262a6b80c3fc47bc8a752ed6c9a1b4cb" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36525 } }
I20260809 04:49:04.641289 26951 leader_election.cc:288] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159), 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883)
W20260809 04:49:04.642246 26472 leader_election.cc:341] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c2e8a79ed2b46eaac833bf2161a258e (127.7.171.3:45159): Not found: Tablet not found: 04c3eb37159e4e43bc28845e9e2759ed
W20260809 04:49:04.642362 26474 leader_election.cc:341] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ea5833ff17d42f5b76c082fe8f99413 (127.7.171.2:45883): Not found: Tablet not found: 04c3eb37159e4e43bc28845e9e2759ed
I20260809 04:49:04.642429 26474 leader_election.cc:302] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262a6b80c3fc47bc8a752ed6c9a1b4cb; no voters: 1c2e8a79ed2b46eaac833bf2161a258e, 5ea5833ff17d42f5b76c082fe8f99413
I20260809 04:49:04.642584 26951 raft_consensus.cc:2726] T 04c3eb37159e4e43bc28845e9e2759ed P 262a6b80c3fc47bc8a752ed6c9a1b4cb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:49:05.084731 26952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:05.084253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.1:34611)
0809 04:49:05.084579 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:05.084633 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":125,"threads_started":1}
I20260809 04:49:05.207665 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:05.207072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.2:35999)
0809 04:49:05.207369 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:05.207512 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":83,"threads_started":1}
I20260809 04:49:05.400792 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 26411
I20260809 04:49:05.408303 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:05.407751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:34927 (local address 127.7.171.3:41619)
0809 04:49:05.408043 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:05.408168 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.62:34927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":85,"threads_started":1}
I20260809 04:49:05.410210 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 26601
I20260809 04:49:05.417732 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 26751
2026-08-09T04:49:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59645 ms)
[----------] 30 tests from SecurityITest (247795 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:05 dist-test-slave-1zqn krb5kdc[26957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:05 dist-test-slave-1zqn krb5kdc[26957](info): set up 2 sockets
Aug 09 04:49:05 dist-test-slave-1zqn krb5kdc[26957](info): commencing operation
krb5kdc: starting...
W20260809 04:49:06.579113 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:06 dist-test-slave-1zqn krb5kdc[26957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:49:06.716238 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43695
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:43695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:06.833055 26970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:06.833393 26970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:06.833480 26970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:06.837600 26970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:06.837702 26970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:06.837745 26970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:06.837806 26970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:06.837847 26970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:06.843377 26970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:43695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:43695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:06.844540 26970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:06.845608 26970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:06.854944 26977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:06.854991 26976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:06.855001 26970 server_base.cc:1034] running on GCE node
W20260809 04:49:06.854925 26979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:06.855698 26970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:06.856832 26970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:06.858031 26970 hybrid_clock.cc:648] HybridClock initialized: now 1786250946857989 us; error 54 us; skew 500 ppm
Aug 09 04:49:06 dist-test-slave-1zqn krb5kdc[26957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250946, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:06.864168 26970 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:49:06.865080 26970 webserver.cc:460] Webserver started at http://127.7.171.62:35357/ using document root <none> and password file <none>
I20260809 04:49:06.865654 26970 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:06.865751 26970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:06.865991 26970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:06.868168 26970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "2232419692ee4658988fc036c508aa7c"
format_stamp: "Formatted at 2026-08-09 04:49:06 on dist-test-slave-1zqn"
server_key: "9d89fdcba0fd2e9fc6c7662a9c8ecd27"
server_key_iv: "621f7f8ba38ca8d903a182d09fca0bc2"
server_key_version: "encryptionkey@0"
I20260809 04:49:06.868743 26970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "2232419692ee4658988fc036c508aa7c"
format_stamp: "Formatted at 2026-08-09 04:49:06 on dist-test-slave-1zqn"
server_key: "9d89fdcba0fd2e9fc6c7662a9c8ecd27"
server_key_iv: "621f7f8ba38ca8d903a182d09fca0bc2"
server_key_version: "encryptionkey@0"
I20260809 04:49:06.872550 26970 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:49:06.874904 26986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:06.876147 26970 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:06.876293 26970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "2232419692ee4658988fc036c508aa7c"
format_stamp: "Formatted at 2026-08-09 04:49:06 on dist-test-slave-1zqn"
server_key: "9d89fdcba0fd2e9fc6c7662a9c8ecd27"
server_key_iv: "621f7f8ba38ca8d903a182d09fca0bc2"
server_key_version: "encryptionkey@0"
I20260809 04:49:06.876405 26970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:06.916011 26970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:06.919649 26970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:06.919871 26970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:06.927735 26970 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:43695
I20260809 04:49:06.927764 27044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:43695 every 8 connection(s)
I20260809 04:49:06.928802 26970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:06.931571 27045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:06.934324 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 26970
I20260809 04:49:06.934428 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:06.934746 7852 external_mini_cluster.cc:1428] Setting key b7a3d7e18ad704b5eced4c00b6a4e70d
I20260809 04:49:06.937687 27045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c: Bootstrap starting.
I20260809 04:49:06.939889 27045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:06.940696 27045 log.cc:824] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:06.942554 27045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c: No bootstrap required, opened a new log
Aug 09 04:49:06 dist-test-slave-1zqn krb5kdc[26957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:06.946352 27045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2232419692ee4658988fc036c508aa7c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43695 } }
I20260809 04:49:06.946676 27045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:06.946740 27045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2232419692ee4658988fc036c508aa7c, State: Initialized, Role: FOLLOWER
I20260809 04:49:06.947289 27045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2232419692ee4658988fc036c508aa7c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43695 } }
I20260809 04:49:06.947458 27045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:06.947544 27045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:06.947649 27045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:06.948789 27045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2232419692ee4658988fc036c508aa7c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43695 } }
I20260809 04:49:06.949136 27045 leader_election.cc:302] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2232419692ee4658988fc036c508aa7c; no voters:
I20260809 04:49:06.949427 27045 leader_election.cc:288] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:06.949537 27051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:06.949779 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:06.936231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49926 (local address 127.7.171.62:43695)
0809 04:49:06.936663 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:06.936679 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:06.936714 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:49:06.937341 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:06.937359 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:06.937682 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:06.938109 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:06.938881 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:06.939885 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:06.940823 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:06.941490 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:06.941664 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:06.944495 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:06.944500 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:06.944514 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:06.944540 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:06.946332 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:06.946872 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:06.946879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:06.946886 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:06.947047 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:06.947381 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:06.947384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:06.947385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:06.947954 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:06.948209 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:06.948448 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 04:49:06.948790 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":130,"threads_started":1}
I20260809 04:49:06.949759 27051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [term 1 LEADER]: Becoming Leader. State: Replica: 2232419692ee4658988fc036c508aa7c, State: Running, Role: LEADER
I20260809 04:49:06.950232 27051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2232419692ee4658988fc036c508aa7c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43695 } }
I20260809 04:49:06.950711 27045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:06.952548 27053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2232419692ee4658988fc036c508aa7c. Latest consensus state: current_term: 1 leader_uuid: "2232419692ee4658988fc036c508aa7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2232419692ee4658988fc036c508aa7c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43695 } } }
I20260809 04:49:06.952925 27053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:06.952817 27052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2232419692ee4658988fc036c508aa7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2232419692ee4658988fc036c508aa7c" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 43695 } } }
I20260809 04:49:06.953425 27052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c [sys.catalog]: This master's current role is: LEADER
W20260809 04:49:06.955816 27066 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:06.955902 27066 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:06.955982 27068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:06.956736 27068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:06.962025 27068 catalog_manager.cc:1347] Generated new cluster ID: 61e9b369e10646cc860de7974616cf1c
I20260809 04:49:06.962154 27068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:06.978415 27068 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:06.979770 27068 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:06.988600 27068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c: Generated new TSK 0
I20260809 04:49:06.989468 27068 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:49:07.030611 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43695
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:07.147238 27073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:07.147555 27073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:07.147629 27073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:07.151858 27073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:07.151943 27073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:07.152086 27073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:49:07.157473 27073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:07.158561 27073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:07.159629 27073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:07.169673 27083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:07.169647 27081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:07.169651 27073 server_base.cc:1034] running on GCE node
W20260809 04:49:07.169615 27080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:07.170248 27073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:07.170979 27073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:07.172142 27073 hybrid_clock.cc:648] HybridClock initialized: now 1786250947172120 us; error 34 us; skew 500 ppm
Aug 09 04:49:07 dist-test-slave-1zqn krb5kdc[26957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:07.175065 27073 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:49:07.175942 27073 webserver.cc:460] Webserver started at http://127.7.171.1:46515/ using document root <none> and password file <none>
I20260809 04:49:07.176595 27073 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:07.176698 27073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:07.176937 27073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:07.179062 27073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "d4a462c70f634ce8b18c32dcae8fb143"
format_stamp: "Formatted at 2026-08-09 04:49:07 on dist-test-slave-1zqn"
server_key: "0b7bab61b0a37ea4254749b437b88e5c"
server_key_iv: "78f9aad8ed699add4638bde3cc8bddbb"
server_key_version: "encryptionkey@0"
I20260809 04:49:07.179735 27073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "d4a462c70f634ce8b18c32dcae8fb143"
format_stamp: "Formatted at 2026-08-09 04:49:07 on dist-test-slave-1zqn"
server_key: "0b7bab61b0a37ea4254749b437b88e5c"
server_key_iv: "78f9aad8ed699add4638bde3cc8bddbb"
server_key_version: "encryptionkey@0"
I20260809 04:49:07.183617 27073 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 04:49:07.186187 27090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:07.187388 27073 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:49:07.187556 27073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "d4a462c70f634ce8b18c32dcae8fb143"
format_stamp: "Formatted at 2026-08-09 04:49:07 on dist-test-slave-1zqn"
server_key: "0b7bab61b0a37ea4254749b437b88e5c"
server_key_iv: "78f9aad8ed699add4638bde3cc8bddbb"
server_key_version: "encryptionkey@0"
I20260809 04:49:07.187673 27073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:07.201308 27073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:07.204463 27073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:07.204679 27073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:07.205286 27073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:07.206167 27073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:07.206238 27073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:07.206311 27073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:07.206364 27073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:07.215899 27073 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:45311
I20260809 04:49:07.215968 27214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:45311 every 8 connection(s)
I20260809 04:49:07.216913 27073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:49:07.218366 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 27073
I20260809 04:49:07.218497 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:49:07.218891 7852 external_mini_cluster.cc:1428] Setting key 2151814b9a89548e0f6d639e1d92a476
Aug 09 04:49:07 dist-test-slave-1zqn krb5kdc[26957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:49:07.232795 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.218878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:58203 (local address 127.7.171.62:43695)
0809 04:49:07.219061 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:07.219068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:07.220072 (+ 1004us) server_negotiation.cc:407] Connection header received
0809 04:49:07.221202 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:07.221205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:07.221258 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:07.221338 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:07.222511 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.223335 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.224741 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.225176 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.225305 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.228466 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:07.228470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:07.228473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:07.228505 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:07.230303 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:07.230877 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:07.230881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:07.230883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:07.230985 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:07.231413 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:07.231416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:07.231418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:07.231639 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:07.231724 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:07.232523 (+ 799us) server_negotiation.cc:299] Negotiation successful
0809 04:49:07.232651 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:49:07.233651 27217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.219288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43695 (local address 127.7.171.1:58203)
0809 04:49:07.219839 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:07.219917 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:07.220995 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:07.221517 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:07.221524 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:07.221883 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:07.222357 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.222370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.223489 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.223492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.224597 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.224608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.225341 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.225345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.225518 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.226249 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:07.226266 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:07.228283 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:07.230454 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:07.230459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:07.230470 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:07.230728 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:07.231109 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:07.231112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:07.231114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:07.231258 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:07.231740 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:07.231745 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:07.232137 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 04:49:07.232339 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 04:49:07.232618 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":145,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
I20260809 04:49:07.235060 27215 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43695
I20260809 04:49:07.235335 27215 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:07.235862 27215 heartbeater.cc:511] Master 127.7.171.62:43695 requested a full tablet report, sending...
I20260809 04:49:07.237566 27001 ts_manager.cc:194] Registered new tserver with Master: d4a462c70f634ce8b18c32dcae8fb143 (127.7.171.1:45311)
I20260809 04:49:07.238819 27001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:58203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:49:07.252414 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43695
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:07.369151 27220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:07.369462 27220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:07.369555 27220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:07.373719 27220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:07.373805 27220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:07.373898 27220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:49:07.379441 27220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:07.380630 27220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:07.381724 27220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:07.392083 27228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:07.392290 27220 server_base.cc:1034] running on GCE node
W20260809 04:49:07.392045 27226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:07.392583 27225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:07.393096 27220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:07.393800 27220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:07.395015 27220 hybrid_clock.cc:648] HybridClock initialized: now 1786250947395002 us; error 30 us; skew 500 ppm
Aug 09 04:49:07 dist-test-slave-1zqn krb5kdc[26957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:07.398222 27220 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:49:07.399237 27220 webserver.cc:460] Webserver started at http://127.7.171.2:36735/ using document root <none> and password file <none>
I20260809 04:49:07.399863 27220 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:07.399966 27220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:07.400249 27220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:07.402468 27220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "1bb1a6fb81164271899c15d2cb10f833"
format_stamp: "Formatted at 2026-08-09 04:49:07 on dist-test-slave-1zqn"
server_key: "d131fba2b96b3dc48bb27e17bed0b6e4"
server_key_iv: "6fe9464224fd7b1028fc6d896f3959e9"
server_key_version: "encryptionkey@0"
I20260809 04:49:07.403081 27220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "1bb1a6fb81164271899c15d2cb10f833"
format_stamp: "Formatted at 2026-08-09 04:49:07 on dist-test-slave-1zqn"
server_key: "d131fba2b96b3dc48bb27e17bed0b6e4"
server_key_iv: "6fe9464224fd7b1028fc6d896f3959e9"
server_key_version: "encryptionkey@0"
I20260809 04:49:07.407155 27220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 04:49:07.409801 27237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:07.411096 27220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:49:07.411230 27220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "1bb1a6fb81164271899c15d2cb10f833"
format_stamp: "Formatted at 2026-08-09 04:49:07 on dist-test-slave-1zqn"
server_key: "d131fba2b96b3dc48bb27e17bed0b6e4"
server_key_iv: "6fe9464224fd7b1028fc6d896f3959e9"
server_key_version: "encryptionkey@0"
I20260809 04:49:07.411343 27220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:07.449924 27220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:07.453677 27220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:07.453894 27220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:07.454548 27220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:07.455428 27220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:07.455499 27220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:07.455565 27220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:07.455616 27220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:07.465133 27220 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:33313
I20260809 04:49:07.465154 27357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:33313 every 8 connection(s)
I20260809 04:49:07.466055 27220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:49:07.470465 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 27220
I20260809 04:49:07.470582 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:49:07.470914 7852 external_mini_cluster.cc:1428] Setting key fb1bd188934117eea198543d94fa9cce
Aug 09 04:49:07 dist-test-slave-1zqn krb5kdc[26957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:07.482389 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.468496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:50357 (local address 127.7.171.62:43695)
0809 04:49:07.468669 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:07.468673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:07.469652 (+ 979us) server_negotiation.cc:407] Connection header received
0809 04:49:07.471235 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:07.471238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:07.471299 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:07.471405 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:07.472821 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.473346 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.474519 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.474755 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.474868 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.477793 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:07.477799 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:07.477803 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:07.477837 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:07.479670 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:07.480496 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:07.480501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:07.480503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:07.480616 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:07.481045 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:07.481050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:07.481054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:07.481320 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:07.481428 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:07.481996 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 04:49:07.482141 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:49:07.483299 27361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.468912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43695 (local address 127.7.171.2:50357)
0809 04:49:07.469487 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:07.469531 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:07.471014 (+ 1483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:07.471587 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:07.471595 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:07.472137 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:07.472674 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.472687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.473478 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.473482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.474388 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.474397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.474910 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.474914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.475093 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.475742 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:07.475759 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:07.477588 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:07.479791 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:07.479796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:07.479808 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:07.480392 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:07.480777 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:07.480780 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:07.480782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:07.480930 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:07.481431 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:07.481437 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:07.481710 (+ 273us) client_negotiation.cc:769] Sending connection context
0809 04:49:07.481876 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 04:49:07.482232 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":137,"threads_started":1}
I20260809 04:49:07.484715 27358 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43695
I20260809 04:49:07.485011 27358 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:07.485569 27358 heartbeater.cc:511] Master 127.7.171.62:43695 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
I20260809 04:49:07.486948 27001 ts_manager.cc:194] Registered new tserver with Master: 1bb1a6fb81164271899c15d2cb10f833 (127.7.171.2:33313)
I20260809 04:49:07.487926 27001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:50357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:49:07.505264 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43695
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:46125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:07.626564 27366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:07.626820 27366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:07.626864 27366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:07.630743 27366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:07.630801 27366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:07.630857 27366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:49:07.636006 27366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:46125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:43695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:07.637006 27366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:07.637957 27366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:07.647951 27373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:07.647972 27372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:07.648294 27366 server_base.cc:1034] running on GCE node
W20260809 04:49:07.647977 27375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:07.648790 27366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:07.649464 27366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:07.650677 27366 hybrid_clock.cc:648] HybridClock initialized: now 1786250947650657 us; error 45 us; skew 500 ppm
Aug 09 04:49:07 dist-test-slave-1zqn krb5kdc[26957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:07.653867 27366 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:49:07.654789 27366 webserver.cc:460] Webserver started at http://127.7.171.3:45551/ using document root <none> and password file <none>
I20260809 04:49:07.655375 27366 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:07.655471 27366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:07.655719 27366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:07.657881 27366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "70ce77182f1d4620a515514a0d7cfe4f"
format_stamp: "Formatted at 2026-08-09 04:49:07 on dist-test-slave-1zqn"
server_key: "8a1c0a40ac8c023aa4c0f9dccbb11110"
server_key_iv: "8580267483e2e1a00d0a62b54b1e0694"
server_key_version: "encryptionkey@0"
I20260809 04:49:07.658450 27366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "70ce77182f1d4620a515514a0d7cfe4f"
format_stamp: "Formatted at 2026-08-09 04:49:07 on dist-test-slave-1zqn"
server_key: "8a1c0a40ac8c023aa4c0f9dccbb11110"
server_key_iv: "8580267483e2e1a00d0a62b54b1e0694"
server_key_version: "encryptionkey@0"
I20260809 04:49:07.662233 27366 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:49:07.664673 27382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:07.665922 27366 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:07.666134 27366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "70ce77182f1d4620a515514a0d7cfe4f"
format_stamp: "Formatted at 2026-08-09 04:49:07 on dist-test-slave-1zqn"
server_key: "8a1c0a40ac8c023aa4c0f9dccbb11110"
server_key_iv: "8580267483e2e1a00d0a62b54b1e0694"
server_key_version: "encryptionkey@0"
I20260809 04:49:07.666246 27366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:07.682628 27366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:07.685930 27366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:07.686136 27366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:07.686755 27366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:07.687606 27366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:07.687678 27366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:07.687750 27366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:07.687801 27366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:07.697512 27366 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:43601
I20260809 04:49:07.697553 27506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:43601 every 8 connection(s)
I20260809 04:49:07.698418 27366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:49:07.703329 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 27366
I20260809 04:49:07.703472 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:49:07.703794 7852 external_mini_cluster.cc:1428] Setting key a036206a86a628108eead3f6e19b3b3a
Aug 09 04:49:07 dist-test-slave-1zqn krb5kdc[26957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250947, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:07.713632 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.700474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:37777 (local address 127.7.171.62:43695)
0809 04:49:07.700627 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:07.700631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:07.701550 (+ 919us) server_negotiation.cc:407] Connection header received
0809 04:49:07.702874 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:07.702878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:07.702939 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:07.703144 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:07.704722 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.705512 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.706853 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.707147 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.707251 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.709909 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:07.709912 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:07.709915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:07.709939 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:07.711442 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:07.711963 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:07.711966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:07.711967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:07.712140 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:07.712502 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:07.712504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:07.712506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:07.712682 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:07.712760 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:07.713348 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 04:49:07.713502 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:49:07.714577 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.700900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:43695 (local address 127.7.171.3:37777)
0809 04:49:07.701377 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:07.701421 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:07.702650 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:07.703321 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:07.703328 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:07.703787 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:07.704498 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.704540 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.705647 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.705651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.706715 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.706725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.707334 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.707337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.707498 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.708220 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:07.708236 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:07.709752 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:07.711572 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:07.711577 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:07.711590 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:07.711868 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:07.712251 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:07.712254 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:07.712256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:07.712410 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:07.712777 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:07.712782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:07.713080 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 04:49:07.713278 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 04:49:07.713606 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":126,"threads_started":1}
I20260809 04:49:07.715945 27507 heartbeater.cc:347] Connected to a master server at 127.7.171.62:43695
I20260809 04:49:07.716269 27507 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:07.716969 27507 heartbeater.cc:511] Master 127.7.171.62:43695 requested a full tablet report, sending...
I20260809 04:49:07.718235 27001 ts_manager.cc:194] Registered new tserver with Master: 70ce77182f1d4620a515514a0d7cfe4f (127.7.171.3:43601)
I20260809 04:49:07.718779 27001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:37777
I20260809 04:49:07.730053 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:49:07.740540 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.732134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49928 (local address 127.7.171.62:43695)
0809 04:49:07.732261 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:07.732272 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:07.732663 (+ 391us) server_negotiation.cc:407] Connection header received
0809 04:49:07.732830 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:07.732833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:07.732890 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:07.732976 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:07.733715 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.734250 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.735140 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.735379 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.735501 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.737033 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:07.737037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:07.737040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:07.737075 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:07.738575 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:07.739078 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:07.739083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:07.739085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:07.739189 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:07.739518 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:07.739522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:07.739524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:07.739779 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:07.739937 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:07.740216 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 04:49:07.740344 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:49:07.743825 27001 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:49:07.746942 27001 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:49:07.763736 27518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.756503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:43601 (local address 127.0.0.1:59146)
0809 04:49:07.757546 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:07.757590 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:07.757744 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:07.758518 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:07.758525 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:07.758553 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:07.758727 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.758734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.760596 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.760600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.761839 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.761851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.763071 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.763075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.763206 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.763272 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 04:49:07.763400 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 04:49:07.763561 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"spinlock_wait_cycles":56064,"thread_start_us":124,"threads_started":1}
I20260809 04:49:07.764163 27519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.756657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59146 (local address 127.7.171.3:43601)
0809 04:49:07.757900 (+ 1243us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:07.757917 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:07.757948 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:49:07.758057 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:07.758063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:07.758318 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:07.758494 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:07.758861 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.760444 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.762042 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.762928 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.763057 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.763223 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:07.763734 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 04:49:07.763854 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1120,"thread_start_us":107,"threads_started":1}
I20260809 04:49:07.764750 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.756969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:45311 (local address 127.0.0.1:39496)
0809 04:49:07.757522 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:07.757581 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:07.757743 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:07.758725 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:07.758728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:07.758745 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:07.758861 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.758876 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.760162 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.760166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.761787 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.761798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.764287 (+ 2489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.764291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.764389 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.764418 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:49:07.764498 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:49:07.764587 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":1,"spinlock_wait_cycles":5376,"thread_start_us":116,"threads_started":1}
I20260809 04:49:07.764962 27523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.757343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:33313 (local address 127.0.0.1:53856)
0809 04:49:07.757594 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:07.757609 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:07.757889 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:07.758667 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:07.758672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:07.758695 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:07.758832 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.758841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.761967 (+ 3126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.761970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.762798 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.762806 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.764538 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.764542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.764640 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.764666 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:49:07.764752 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:49:07.764825 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"spinlock_wait_cycles":4992,"thread_start_us":120,"threads_started":1}
I20260809 04:49:07.765101 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.757150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39496 (local address 127.7.171.1:45311)
0809 04:49:07.758100 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:07.758108 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:07.758135 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:49:07.758217 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:07.758221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:07.758378 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:07.758560 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:07.758994 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.759996 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.761971 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.762928 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.763057 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.763223 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:07.764774 (+ 1551us) server_negotiation.cc:299] Negotiation successful
0809 04:49:07.764908 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":121,"threads_started":1}
I20260809 04:49:07.765784 27525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.757828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53856 (local address 127.7.171.2:33313)
0809 04:49:07.758174 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:07.758186 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:07.758209 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:49:07.758281 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:07.758286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:07.758477 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:07.759000 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:07.759171 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.760419 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.763399 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.764387 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.764516 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.764670 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:07.765493 (+ 823us) server_negotiation.cc:299] Negotiation successful
0809 04:49:07.765614 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":119,"threads_started":1}
I20260809 04:49:07.767517 27432 tablet_service.cc:1467] Processing CreateTablet for tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad (DEFAULT_TABLE table=test-table [id=5ffc35345075417d941192c8edfc1881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:07.768165 27143 tablet_service.cc:1467] Processing CreateTablet for tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad (DEFAULT_TABLE table=test-table [id=5ffc35345075417d941192c8edfc1881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:07.768751 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:07.768730 27289 tablet_service.cc:1467] Processing CreateTablet for tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad (DEFAULT_TABLE table=test-table [id=5ffc35345075417d941192c8edfc1881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:07.769340 27143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:07.769820 27289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:07.776489 27526 tablet_bootstrap.cc:492] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f: Bootstrap starting.
I20260809 04:49:07.778072 27527 tablet_bootstrap.cc:492] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143: Bootstrap starting.
I20260809 04:49:07.778110 27528 tablet_bootstrap.cc:492] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833: Bootstrap starting.
I20260809 04:49:07.779165 27526 tablet_bootstrap.cc:654] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:07.780385 27527 tablet_bootstrap.cc:654] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:07.780689 27526 log.cc:824] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:07.781327 27528 tablet_bootstrap.cc:654] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:07.781358 27527 log.cc:824] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:07.782250 27528 log.cc:824] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:07.783125 27526 tablet_bootstrap.cc:492] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f: No bootstrap required, opened a new log
I20260809 04:49:07.783349 27526 ts_tablet_manager.cc:1397] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 04:49:07.783571 27527 tablet_bootstrap.cc:492] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143: No bootstrap required, opened a new log
I20260809 04:49:07.783782 27527 ts_tablet_manager.cc:1397] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:49:07.784534 27528 tablet_bootstrap.cc:492] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833: No bootstrap required, opened a new log
I20260809 04:49:07.784722 27528 ts_tablet_manager.cc:1397] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 04:49:07.787801 27526 raft_consensus.cc:348] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70ce77182f1d4620a515514a0d7cfe4f" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 43601 } } peers { permanent_uuid: "1bb1a6fb81164271899c15d2cb10f833" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 33313 } } peers { permanent_uuid: "d4a462c70f634ce8b18c32dcae8fb143" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45311 } }
I20260809 04:49:07.788146 27526 raft_consensus.cc:374] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:07.788244 27526 raft_consensus.cc:729] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70ce77182f1d4620a515514a0d7cfe4f, State: Initialized, Role: FOLLOWER
I20260809 04:49:07.788782 27526 consensus_queue.cc:260] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70ce77182f1d4620a515514a0d7cfe4f" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 43601 } } peers { permanent_uuid: "1bb1a6fb81164271899c15d2cb10f833" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 33313 } } peers { permanent_uuid: "d4a462c70f634ce8b18c32dcae8fb143" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45311 } }
I20260809 04:49:07.788815 27527 raft_consensus.cc:348] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70ce77182f1d4620a515514a0d7cfe4f" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 43601 } } peers { permanent_uuid: "1bb1a6fb81164271899c15d2cb10f833" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 33313 } } peers { permanent_uuid: "d4a462c70f634ce8b18c32dcae8fb143" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45311 } }
I20260809 04:49:07.789130 27527 raft_consensus.cc:374] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:07.789218 27527 raft_consensus.cc:729] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4a462c70f634ce8b18c32dcae8fb143, State: Initialized, Role: FOLLOWER
I20260809 04:49:07.789213 27528 raft_consensus.cc:348] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70ce77182f1d4620a515514a0d7cfe4f" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 43601 } } peers { permanent_uuid: "1bb1a6fb81164271899c15d2cb10f833" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 33313 } } peers { permanent_uuid: "d4a462c70f634ce8b18c32dcae8fb143" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45311 } }
I20260809 04:49:07.789537 27528 raft_consensus.cc:374] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:07.789604 27528 raft_consensus.cc:729] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bb1a6fb81164271899c15d2cb10f833, State: Initialized, Role: FOLLOWER
I20260809 04:49:07.789674 27527 consensus_queue.cc:260] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70ce77182f1d4620a515514a0d7cfe4f" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 43601 } } peers { permanent_uuid: "1bb1a6fb81164271899c15d2cb10f833" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 33313 } } peers { permanent_uuid: "d4a462c70f634ce8b18c32dcae8fb143" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45311 } }
I20260809 04:49:07.790012 27526 ts_tablet_manager.cc:1428] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 04:49:07.790021 27528 consensus_queue.cc:260] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70ce77182f1d4620a515514a0d7cfe4f" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 43601 } } peers { permanent_uuid: "1bb1a6fb81164271899c15d2cb10f833" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 33313 } } peers { permanent_uuid: "d4a462c70f634ce8b18c32dcae8fb143" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45311 } }
I20260809 04:49:07.790406 27507 heartbeater.cc:503] Master 127.7.171.62:43695 was elected leader, sending a full tablet report...
I20260809 04:49:07.790846 27527 ts_tablet_manager.cc:1428] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 04:49:07.791103 27528 ts_tablet_manager.cc:1428] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:49:07.791205 27215 heartbeater.cc:503] Master 127.7.171.62:43695 was elected leader, sending a full tablet report...
I20260809 04:49:07.791467 27358 heartbeater.cc:503] Master 127.7.171.62:43695 was elected leader, sending a full tablet report...
I20260809 04:49:07.882059 27533 raft_consensus.cc:482] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:49:07.882334 27533 raft_consensus.cc:504] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70ce77182f1d4620a515514a0d7cfe4f" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 43601 } } peers { permanent_uuid: "1bb1a6fb81164271899c15d2cb10f833" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 33313 } } peers { permanent_uuid: "d4a462c70f634ce8b18c32dcae8fb143" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45311 } }
I20260809 04:49:07.883674 27533 leader_election.cc:288] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70ce77182f1d4620a515514a0d7cfe4f (127.7.171.3:43601), 1bb1a6fb81164271899c15d2cb10f833 (127.7.171.2:33313)
I20260809 04:49:07.888325 27535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.883934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:43601 (local address 127.7.171.1:40353)
0809 04:49:07.884325 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:07.884338 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:07.884473 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:07.884959 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:07.884963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:07.884984 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:07.885151 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.885162 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.886083 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.886086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.886986 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.886997 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.887835 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.887839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.887948 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.887970 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:49:07.888047 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 04:49:07.888154 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":180,"threads_started":1}
I20260809 04:49:07.888360 27519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.884010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:40353 (local address 127.7.171.3:43601)
0809 04:49:07.884216 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:07.884221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:07.884427 (+ 206us) server_negotiation.cc:407] Connection header received
0809 04:49:07.884646 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:07.884650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:07.884708 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:07.884762 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:07.885283 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.885958 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.887144 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.887701 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.887792 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.887853 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:07.888132 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 04:49:07.888218 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260809 04:49:07.888526 27525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.884655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:48963 (local address 127.7.171.2:33313)
0809 04:49:07.884791 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:07.884795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:07.884884 (+ 89us) server_negotiation.cc:407] Connection header received
0809 04:49:07.885059 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:07.885064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:07.885122 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:07.885213 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:07.885572 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.886126 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.887384 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.887828 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.887935 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.887999 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:07.888284 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 04:49:07.888376 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:49:07.888587 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.884463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:33313 (local address 127.7.171.1:48963)
0809 04:49:07.884804 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:07.884817 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:07.884915 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:07.885259 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:07.885263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:07.885282 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:07.885406 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.885412 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.886243 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.886245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.887245 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:07.887257 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.887960 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.887964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:07.888143 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.888169 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:49:07.888257 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:49:07.888330 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":119,"threads_started":1}
I20260809 04:49:07.889571 27310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d11bef0be2e480daa1d6ca7b4e9e1ad" candidate_uuid: "d4a462c70f634ce8b18c32dcae8fb143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bb1a6fb81164271899c15d2cb10f833" is_pre_election: true
I20260809 04:49:07.889864 27310 raft_consensus.cc:2454] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4a462c70f634ce8b18c32dcae8fb143 in term 0.
I20260809 04:49:07.890413 27091 leader_election.cc:302] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bb1a6fb81164271899c15d2cb10f833, d4a462c70f634ce8b18c32dcae8fb143; no voters:
I20260809 04:49:07.890434 27456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d11bef0be2e480daa1d6ca7b4e9e1ad" candidate_uuid: "d4a462c70f634ce8b18c32dcae8fb143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70ce77182f1d4620a515514a0d7cfe4f" is_pre_election: true
I20260809 04:49:07.890729 27456 raft_consensus.cc:2454] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4a462c70f634ce8b18c32dcae8fb143 in term 0.
I20260809 04:49:07.890746 27533 raft_consensus.cc:2781] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:49:07.890866 27533 raft_consensus.cc:482] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:49:07.891026 27533 raft_consensus.cc:3037] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:07.892241 27533 raft_consensus.cc:504] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70ce77182f1d4620a515514a0d7cfe4f" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 43601 } } peers { permanent_uuid: "1bb1a6fb81164271899c15d2cb10f833" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 33313 } } peers { permanent_uuid: "d4a462c70f634ce8b18c32dcae8fb143" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45311 } }
I20260809 04:49:07.892542 27533 leader_election.cc:288] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [CANDIDATE]: Term 1 election: Requested vote from peers 70ce77182f1d4620a515514a0d7cfe4f (127.7.171.3:43601), 1bb1a6fb81164271899c15d2cb10f833 (127.7.171.2:33313)
I20260809 04:49:07.892946 27456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d11bef0be2e480daa1d6ca7b4e9e1ad" candidate_uuid: "d4a462c70f634ce8b18c32dcae8fb143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70ce77182f1d4620a515514a0d7cfe4f"
I20260809 04:49:07.893077 27456 raft_consensus.cc:3037] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:07.893164 27310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d11bef0be2e480daa1d6ca7b4e9e1ad" candidate_uuid: "d4a462c70f634ce8b18c32dcae8fb143" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bb1a6fb81164271899c15d2cb10f833"
I20260809 04:49:07.893328 27310 raft_consensus.cc:3037] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:07.894220 27456 raft_consensus.cc:2454] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4a462c70f634ce8b18c32dcae8fb143 in term 1.
I20260809 04:49:07.894462 27310 raft_consensus.cc:2454] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4a462c70f634ce8b18c32dcae8fb143 in term 1.
I20260809 04:49:07.894516 27091 leader_election.cc:302] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70ce77182f1d4620a515514a0d7cfe4f, d4a462c70f634ce8b18c32dcae8fb143; no voters:
I20260809 04:49:07.894692 27533 raft_consensus.cc:2781] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:07.895062 27533 raft_consensus.cc:686] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [term 1 LEADER]: Becoming Leader. State: Replica: d4a462c70f634ce8b18c32dcae8fb143, State: Running, Role: LEADER
I20260809 04:49:07.895416 27533 consensus_queue.cc:237] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70ce77182f1d4620a515514a0d7cfe4f" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 43601 } } peers { permanent_uuid: "1bb1a6fb81164271899c15d2cb10f833" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 33313 } } peers { permanent_uuid: "d4a462c70f634ce8b18c32dcae8fb143" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45311 } }
I20260809 04:49:07.898972 27003 catalog_manager.cc:5495] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4a462c70f634ce8b18c32dcae8fb143 (127.7.171.1). New cstate: current_term: 1 leader_uuid: "d4a462c70f634ce8b18c32dcae8fb143" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70ce77182f1d4620a515514a0d7cfe4f" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 43601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bb1a6fb81164271899c15d2cb10f833" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 33313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4a462c70f634ce8b18c32dcae8fb143" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 45311 } health_report { overall_health: HEALTHY } } }
I20260809 04:49:07.917675 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.913922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39512 (local address 127.7.171.1:45311)
0809 04:49:07.914086 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:07.914090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:07.914104 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:49:07.914239 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:07.914241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:07.914299 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:07.914391 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:07.914656 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.915197 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.916238 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:07.916464 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:07.916559 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:07.916810 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:07.917252 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:07.917348 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:07.917471 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 04:49:07.917525 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:49:07.923658 27310 raft_consensus.cc:1261] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833 [term 1 FOLLOWER]: Refusing update from remote peer d4a462c70f634ce8b18c32dcae8fb143: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:07.923659 27456 raft_consensus.cc:1261] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f [term 1 FOLLOWER]: Refusing update from remote peer d4a462c70f634ce8b18c32dcae8fb143: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:07.924391 27533 consensus_queue.cc:1035] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70ce77182f1d4620a515514a0d7cfe4f" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:07.924561 27537 consensus_queue.cc:1035] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P d4a462c70f634ce8b18c32dcae8fb143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bb1a6fb81164271899c15d2cb10f833" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 33313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:07.934783 27542 mvcc.cc:205] Tried to move back new op lower bound from 7316483882688086016 to 7316483882582704128. Current Snapshot: MvccSnapshot[applied={T|T < 7316483882688086016}]
I20260809 04:49:07.934983 27541 mvcc.cc:205] Tried to move back new op lower bound from 7316483882688086016 to 7316483882582704128. Current Snapshot: MvccSnapshot[applied={T|T < 7316483882688086016}]
I20260809 04:49:07.936159 27540 mvcc.cc:205] Tried to move back new op lower bound from 7316483882688086016 to 7316483882582704128. Current Snapshot: MvccSnapshot[applied={T|T < 7316483882688086016}]
I20260809 04:49:07.944763 27003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:07.944933 27003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:07.947279 27003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2232419692ee4658988fc036c508aa7c: Sending DeleteTablet for 3 replicas of tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad
I20260809 04:49:07.948123 27143 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:07 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:39496
I20260809 04:49:07.948123 27289 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:07 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:53856
I20260809 04:49:07.948798 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 27073
I20260809 04:49:07.948818 27432 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:07 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:59146
I20260809 04:49:07.950906 27551 tablet_replica.cc:330] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f: stopping tablet replica
I20260809 04:49:07.951277 27551 raft_consensus.cc:2229] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:49:07.951660 27551 raft_consensus.cc:2258] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:07.952046 27550 tablet_replica.cc:330] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833: stopping tablet replica
I20260809 04:49:07.952363 27550 raft_consensus.cc:2229] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:49:07.952634 27550 raft_consensus.cc:2258] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:07.954208 27551 ts_tablet_manager.cc:1905] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:07.954208 27550 ts_tablet_manager.cc:1905] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:07.956987 27550 ts_tablet_manager.cc:1918] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:07.956987 27551 ts_tablet_manager.cc:1918] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:07.957096 27551 log.cc:1196] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/7d11bef0be2e480daa1d6ca7b4e9e1ad
I20260809 04:49:07.957096 27550 log.cc:1196] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/7d11bef0be2e480daa1d6ca7b4e9e1ad
I20260809 04:49:07.957448 27550 ts_tablet_manager.cc:1939] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 1bb1a6fb81164271899c15d2cb10f833: Deleting consensus metadata
I20260809 04:49:07.957471 27551 ts_tablet_manager.cc:1939] T 7d11bef0be2e480daa1d6ca7b4e9e1ad P 70ce77182f1d4620a515514a0d7cfe4f: Deleting consensus metadata
W20260809 04:49:07.958302 26988 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:45311 (error 108)
I20260809 04:49:07.958943 26987 catalog_manager.cc:4841] TS 70ce77182f1d4620a515514a0d7cfe4f (127.7.171.3:43601): tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad (table test-table [id=5ffc35345075417d941192c8edfc1881]) successfully deleted
I20260809 04:49:07.959160 26987 catalog_manager.cc:4841] TS 1bb1a6fb81164271899c15d2cb10f833 (127.7.171.2:33313): tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad (table test-table [id=5ffc35345075417d941192c8edfc1881]) successfully deleted
I20260809 04:49:07.959309 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 27220
I20260809 04:49:07.959496 27520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:07.959075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:45311 (local address 127.0.0.1:39518)
0809 04:49:07.959290 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:07.959379 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:45311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260809 04:49:07.959955 26988 catalog_manager.cc:4584] TS d4a462c70f634ce8b18c32dcae8fb143 (127.7.171.1:45311): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d11bef0be2e480daa1d6ca7b4e9e1ad: Network error: Client connection negotiation failed: client connection to 127.7.171.1:45311: connect: Connection refused (error 111)
I20260809 04:49:07.966697 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 27366
I20260809 04:49:07.975059 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 26970
2026-08-09T04:49:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2558 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T04:49:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:08Z Disabled control of system clock
I20260809 04:49:08.010538 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35345
--webserver_interface=127.7.171.62
--webserver_port=0
--builtin_ntp_servers=127.7.171.20:46599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:35345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:49:08.125626 27556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:08.125998 27556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:08.126060 27556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:08.130015 27556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:08.130065 27556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:08.130081 27556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:08.130098 27556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:08.130154 27556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:08.134828 27556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:46599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:35345
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:08.135880 27556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:08.137329 27556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:08.146365 27564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:08.146374 27562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:08.146374 27561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:08.146431 27556 server_base.cc:1034] running on GCE node
I20260809 04:49:08.147197 27556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:08.148388 27556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:08.149607 27556 hybrid_clock.cc:648] HybridClock initialized: now 1786250948149575 us; error 47 us; skew 500 ppm
I20260809 04:49:08.151657 27556 webserver.cc:460] Webserver started at http://127.7.171.62:45739/ using document root <none> and password file <none>
I20260809 04:49:08.152323 27556 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:08.152418 27556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:08.152658 27556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:08.154748 27556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "f0e7949bab8547b8a6c63ae36db65ef7"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "c316846127d89e897e120ba7158b496f"
server_key_iv: "bb40190091fb4353096963e42f210b83"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.155349 27556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "f0e7949bab8547b8a6c63ae36db65ef7"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "c316846127d89e897e120ba7158b496f"
server_key_iv: "bb40190091fb4353096963e42f210b83"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.159286 27556 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:49:08.161813 27570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:08.163146 27556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:49:08.163297 27556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "f0e7949bab8547b8a6c63ae36db65ef7"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "c316846127d89e897e120ba7158b496f"
server_key_iv: "bb40190091fb4353096963e42f210b83"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.163410 27556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:08.182173 27556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:08.182922 27556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:08.183163 27556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:08.190901 27556 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:35345
I20260809 04:49:08.190907 27624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:35345 every 8 connection(s)
I20260809 04:49:08.191903 27556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:08.195267 27625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:08.197216 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 27556
I20260809 04:49:08.197360 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:08.197664 7852 external_mini_cluster.cc:1428] Setting key e93cae4b0df2b4a35438218d3fa16345
I20260809 04:49:08.202345 27625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7: Bootstrap starting.
I20260809 04:49:08.204752 27625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:08.205511 27625 log.cc:824] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:08.207571 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.199210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39794 (local address 127.7.171.62:35345)
0809 04:49:08.199945 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:08.199962 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:08.200000 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:49:08.200752 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:08.200769 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:08.201216 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:08.201562 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:08.201989 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.203076 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.203964 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.204512 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.204688 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.205571 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:08.205576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:08.205590 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:08.206105 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:08.206178 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:08.206207 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:08.206315 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:08.206527 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 04:49:08.206817 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":142,"threads_started":1}
I20260809 04:49:08.207659 27625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7: No bootstrap required, opened a new log
I20260809 04:49:08.211229 27625 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e7949bab8547b8a6c63ae36db65ef7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35345 } }
I20260809 04:49:08.211498 27625 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:08.211593 27625 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e7949bab8547b8a6c63ae36db65ef7, State: Initialized, Role: FOLLOWER
I20260809 04:49:08.212175 27625 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e7949bab8547b8a6c63ae36db65ef7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35345 } }
I20260809 04:49:08.212347 27625 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:08.212435 27625 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:08.212533 27625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:08.213510 27625 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e7949bab8547b8a6c63ae36db65ef7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35345 } }
I20260809 04:49:08.213860 27625 leader_election.cc:302] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0e7949bab8547b8a6c63ae36db65ef7; no voters:
I20260809 04:49:08.214234 27625 leader_election.cc:288] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:08.214385 27631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:08.214715 27631 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [term 1 LEADER]: Becoming Leader. State: Replica: f0e7949bab8547b8a6c63ae36db65ef7, State: Running, Role: LEADER
I20260809 04:49:08.215085 27631 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e7949bab8547b8a6c63ae36db65ef7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35345 } }
I20260809 04:49:08.215610 27625 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:08.216893 27632 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0e7949bab8547b8a6c63ae36db65ef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e7949bab8547b8a6c63ae36db65ef7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35345 } } }
I20260809 04:49:08.217031 27632 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:08.217314 27633 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0e7949bab8547b8a6c63ae36db65ef7. Latest consensus state: current_term: 1 leader_uuid: "f0e7949bab8547b8a6c63ae36db65ef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e7949bab8547b8a6c63ae36db65ef7" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35345 } } }
I20260809 04:49:08.217412 27633 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:08.217455 27641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:08.220333 27641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:08.225816 27641 catalog_manager.cc:1347] Generated new cluster ID: ee5e589b53a1440c8fcdb4593dfa4f54
I20260809 04:49:08.225914 27641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:08.232868 27641 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:08.233868 27641 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:08.237592 27641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7: Generated new TSK 0
I20260809 04:49:08.238363 27641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:08.251250 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35345
--builtin_ntp_servers=127.7.171.20:46599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:49:08.368760 27652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:08.369029 27652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:08.369096 27652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:08.373157 27652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:08.373236 27652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:08.373323 27652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:49:08.378033 27652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:46599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:08.379093 27652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:08.380589 27652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:08.390033 27657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:08.390275 27652 server_base.cc:1034] running on GCE node
W20260809 04:49:08.390197 27658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:08.390125 27660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:08.390807 27652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:08.391415 27652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:08.392616 27652 hybrid_clock.cc:648] HybridClock initialized: now 1786250948392575 us; error 50 us; skew 500 ppm
I20260809 04:49:08.394381 27652 webserver.cc:460] Webserver started at http://127.7.171.1:43697/ using document root <none> and password file <none>
I20260809 04:49:08.394970 27652 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:08.395064 27652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:08.395305 27652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:08.397436 27652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "aa28cd52a8c9481caf518b9cc62a4ed8"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "a2444b39159a66997bd2dc5756ce2e23"
server_key_iv: "a97160e51e87c67f9cbe2ed2ed8bc0dd"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.398012 27652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "aa28cd52a8c9481caf518b9cc62a4ed8"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "a2444b39159a66997bd2dc5756ce2e23"
server_key_iv: "a97160e51e87c67f9cbe2ed2ed8bc0dd"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.401932 27652 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:49:08.404385 27668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:08.405584 27652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:49:08.405740 27652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "aa28cd52a8c9481caf518b9cc62a4ed8"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "a2444b39159a66997bd2dc5756ce2e23"
server_key_iv: "a97160e51e87c67f9cbe2ed2ed8bc0dd"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.405853 27652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:08.425446 27652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:08.426172 27652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:08.426378 27652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:08.427006 27652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:08.427979 27652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:08.428054 27652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:08.428175 27652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:08.428231 27652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:08.438215 27652 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:34789
I20260809 04:49:08.438233 27788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:34789 every 8 connection(s)
I20260809 04:49:08.439180 27652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:49:08.448693 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.441151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:48217 (local address 127.7.171.62:35345)
0809 04:49:08.441331 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:08.441337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:08.442456 (+ 1119us) server_negotiation.cc:407] Connection header received
0809 04:49:08.443870 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:08.443874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:08.443928 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:08.444012 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:08.444958 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.445523 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.446653 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.446889 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.446977 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.448185 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:08.448188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:08.448190 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:08.448202 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:08.448221 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:08.448229 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:08.448291 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:08.448505 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 04:49:08.448563 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:49:08.449333 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 27652
I20260809 04:49:08.449471 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:49:08.449791 7852 external_mini_cluster.cc:1428] Setting key 886e61133fb04cb351f8f67d7ce40409
I20260809 04:49:08.449872 27791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.441576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35345 (local address 127.7.171.1:48217)
0809 04:49:08.442293 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:08.442336 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:08.443660 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:08.444236 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:08.444242 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:08.444326 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:08.444832 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.444844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.445638 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.445641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.446535 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.446544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.447080 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.447083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.447233 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.447841 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:49:08.447843 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:08.447882 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:49:08.447949 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:08.448336 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:08.448345 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:08.448350 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:49:08.448499 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 04:49:08.448728 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":123,"threads_started":1}
I20260809 04:49:08.451156 27789 heartbeater.cc:347] Connected to a master server at 127.7.171.62:35345
I20260809 04:49:08.451418 27789 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:08.452020 27789 heartbeater.cc:511] Master 127.7.171.62:35345 requested a full tablet report, sending...
I20260809 04:49:08.452093 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35345
--builtin_ntp_servers=127.7.171.20:46599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:49:08.453665 27587 ts_manager.cc:194] Registered new tserver with Master: aa28cd52a8c9481caf518b9cc62a4ed8 (127.7.171.1:34789)
I20260809 04:49:08.455379 27587 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.1:48217
W20260809 04:49:08.566306 27792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:08.566562 27792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:08.566599 27792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:08.570539 27792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:08.570602 27792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:08.570655 27792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:49:08.575251 27792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:46599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:08.576457 27792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:08.577801 27792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:08.587738 27800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:08.587893 27792 server_base.cc:1034] running on GCE node
W20260809 04:49:08.587864 27802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:08.587714 27798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:08.588613 27792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:08.589366 27792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:08.590587 27792 hybrid_clock.cc:648] HybridClock initialized: now 1786250948590553 us; error 47 us; skew 500 ppm
I20260809 04:49:08.592628 27792 webserver.cc:460] Webserver started at http://127.7.171.2:42157/ using document root <none> and password file <none>
I20260809 04:49:08.593230 27792 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:08.593325 27792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:08.593576 27792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:08.595715 27792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "3ed3c3d6e2d5c8904dec6849d6e4e4b5"
server_key_iv: "9727707bea69e88c9753bd6cd86bc2cc"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.596328 27792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "3ed3c3d6e2d5c8904dec6849d6e4e4b5"
server_key_iv: "9727707bea69e88c9753bd6cd86bc2cc"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.600241 27792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:49:08.602905 27808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:08.604341 27792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:08.604475 27792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "3ed3c3d6e2d5c8904dec6849d6e4e4b5"
server_key_iv: "9727707bea69e88c9753bd6cd86bc2cc"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.604595 27792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:08.625114 27792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:08.625872 27792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:08.626092 27792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:08.626730 27792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:08.627720 27792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:08.627794 27792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:08.627868 27792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:08.627920 27792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:08.637714 27792 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:37467
I20260809 04:49:08.637741 27930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:37467 every 8 connection(s)
I20260809 04:49:08.638618 27792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:49:08.640269 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 27792
I20260809 04:49:08.640419 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:49:08.640772 7852 external_mini_cluster.cc:1428] Setting key 14f9e9fcc8ffe2ba67c64263fccece9f
I20260809 04:49:08.642997 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35345
--builtin_ntp_servers=127.7.171.20:46599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:49:08.649503 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.640576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:46933 (local address 127.7.171.62:35345)
0809 04:49:08.640742 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:08.640747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:08.642062 (+ 1315us) server_negotiation.cc:407] Connection header received
0809 04:49:08.643665 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:08.643668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:08.643722 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:08.643803 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:08.644873 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.645400 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.647185 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.647400 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.647483 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.648954 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:08.648958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:08.649010 (+ 52us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:08.649023 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:08.649044 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:08.649053 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:08.649126 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:08.649329 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 04:49:08.649386 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:49:08.650848 27933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.641074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35345 (local address 127.7.171.2:46933)
0809 04:49:08.641826 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:08.641909 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:08.643445 (+ 1536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:08.643976 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:08.643983 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:08.644117 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:08.644674 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.644696 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.645540 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.645546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.647043 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.647058 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.647585 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.647593 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.647823 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.648601 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:49:08.648604 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:08.648666 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:49:08.648761 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:08.649156 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:08.649169 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:08.649176 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 04:49:08.649328 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 04:49:08.649678 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":122,"threads_started":1}
I20260809 04:49:08.652179 27931 heartbeater.cc:347] Connected to a master server at 127.7.171.62:35345
I20260809 04:49:08.652472 27931 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:08.653061 27931 heartbeater.cc:511] Master 127.7.171.62:35345 requested a full tablet report, sending...
I20260809 04:49:08.654290 27587 ts_manager.cc:194] Registered new tserver with Master: bdf9342ca6f2453bb4d6ea564bbdda4e (127.7.171.2:37467)
I20260809 04:49:08.654901 27587 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.2:46933
W20260809 04:49:08.760622 27934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:08.760882 27934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:08.760921 27934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:08.764972 27934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:08.765030 27934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:08.765085 27934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:49:08.770025 27934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:46599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:08.771116 27934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:08.772634 27934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:08.782701 27942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:08.782702 27946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:08.782903 27934 server_base.cc:1034] running on GCE node
W20260809 04:49:08.782774 27943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:08.783535 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:08.784222 27934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:08.785435 27934 hybrid_clock.cc:648] HybridClock initialized: now 1786250948785404 us; error 64 us; skew 500 ppm
I20260809 04:49:08.787461 27934 webserver.cc:460] Webserver started at http://127.7.171.3:42377/ using document root <none> and password file <none>
I20260809 04:49:08.788107 27934 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:08.788208 27934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:08.788465 27934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:08.790535 27934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "3dbe08859bcf4ac5861d37d2b1fcb151"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "d2c6cef6464d6597b7431b2bd4d04194"
server_key_iv: "5e34173d00e883d50976a601bc99dd4e"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.791144 27934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "3dbe08859bcf4ac5861d37d2b1fcb151"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "d2c6cef6464d6597b7431b2bd4d04194"
server_key_iv: "5e34173d00e883d50976a601bc99dd4e"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.795107 27934 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 04:49:08.797693 27952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:08.799037 27934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:49:08.799204 27934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "3dbe08859bcf4ac5861d37d2b1fcb151"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1zqn"
server_key: "d2c6cef6464d6597b7431b2bd4d04194"
server_key_iv: "5e34173d00e883d50976a601bc99dd4e"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.799328 27934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:08.815053 27934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:08.815800 27934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:08.816118 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:08.816843 27934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:08.817971 27934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:08.818053 27934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:08.818123 27934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:08.818176 27934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:08.827150 27934 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:42209
I20260809 04:49:08.827180 28074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:42209 every 8 connection(s)
I20260809 04:49:08.828099 27934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:49:08.831089 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 27934
I20260809 04:49:08.831203 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:49:08.831477 7852 external_mini_cluster.cc:1428] Setting key f8ece4dc6c674fbd9d693101fefa6bbe
I20260809 04:49:08.837807 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.829960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:38265 (local address 127.7.171.62:35345)
0809 04:49:08.830138 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:08.830142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:08.831268 (+ 1126us) server_negotiation.cc:407] Connection header received
0809 04:49:08.832948 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:08.832951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:08.833019 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:08.833123 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:08.834122 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.834662 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.835813 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.836099 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.836218 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.837245 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:08.837248 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:08.837251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:08.837264 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:08.837292 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:08.837300 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:08.837376 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:08.837596 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 04:49:08.837662 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 04:49:08.838658 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.830363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35345 (local address 127.7.171.3:38265)
0809 04:49:08.831039 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:08.831084 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:08.832734 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:08.833304 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:08.833311 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:08.833397 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:08.833975 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.833990 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.834806 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.834810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.835684 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.835693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.836258 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.836262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.836425 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.836993 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:49:08.836994 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:08.837035 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:49:08.837098 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:08.837390 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:08.837402 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:08.837412 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 04:49:08.837593 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 04:49:08.837869 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":125,"threads_started":1}
I20260809 04:49:08.839795 28075 heartbeater.cc:347] Connected to a master server at 127.7.171.62:35345
I20260809 04:49:08.840032 28075 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:08.840591 28075 heartbeater.cc:511] Master 127.7.171.62:35345 requested a full tablet report, sending...
I20260809 04:49:08.841609 27587 ts_manager.cc:194] Registered new tserver with Master: 3dbe08859bcf4ac5861d37d2b1fcb151 (127.7.171.3:42209)
I20260809 04:49:08.842232 27587 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.3:38265
I20260809 04:49:08.845477 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:49:08.851197 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.847462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39796 (local address 127.7.171.62:35345)
0809 04:49:08.847595 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:08.847599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:08.847817 (+ 218us) server_negotiation.cc:407] Connection header received
0809 04:49:08.847913 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:08.847915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:08.847954 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:08.848103 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:08.848364 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.848897 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.849742 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.849956 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.850086 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.850804 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:08.850807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:08.850809 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:08.850820 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:08.850839 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:08.850847 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:08.850905 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:08.851039 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 04:49:08.851090 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:49:08.854101 27587 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:49:08.857213 27587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:49:08.871644 28093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.865828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:42209 (local address 127.0.0.1:53186)
0809 04:49:08.866799 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:08.866851 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:08.866987 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:08.867803 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:08.867813 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:08.867842 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:08.867997 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.868007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.869072 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.869078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.870241 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.870251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.871082 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.871085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.871207 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.871245 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 04:49:08.871353 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 04:49:08.871473 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"thread_start_us":68,"threads_started":1}
I20260809 04:49:08.871649 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.865978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53186 (local address 127.7.171.3:42209)
0809 04:49:08.867291 (+ 1313us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:08.867299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:08.867320 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:49:08.867382 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:08.867386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:08.867629 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:08.867779 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:08.868127 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.868940 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.870373 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.870945 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.871078 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.871287 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:08.871378 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 04:49:08.871473 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1215,"thread_start_us":174,"threads_started":1}
I20260809 04:49:08.871969 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.865869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:34789 (local address 127.0.0.1:52600)
0809 04:49:08.867076 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:08.867091 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:08.867181 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:08.867905 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:08.867908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:08.867921 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:08.868034 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.868118 (+ 84us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.869644 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.869646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.870474 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.870483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.871589 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.871593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.871680 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.871704 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:49:08.871750 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 04:49:08.871832 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1138,"mutex_wait_us":14,"thread_start_us":108,"threads_started":1}
I20260809 04:49:08.872649 28095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.866669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52600 (local address 127.7.171.1:34789)
0809 04:49:08.867326 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:08.867338 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:08.867360 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:49:08.867447 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:08.867519 (+ 72us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:08.867752 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:08.868275 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:08.868431 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.869522 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.870657 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.871451 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.872120 (+ 669us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.872301 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:08.872359 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 04:49:08.872477 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":131,"threads_started":1}
I20260809 04:49:08.873656 28089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.866202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:37467 (local address 127.0.0.1:42832)
0809 04:49:08.867360 (+ 1158us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:08.867374 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:08.867532 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:08.867965 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:08.867967 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:08.867985 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:08.868157 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.868166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.869629 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.869634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.871748 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:08.871757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.873218 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.873221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:08.873307 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.873329 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:49:08.873437 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 04:49:08.873529 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"mutex_wait_us":44,"thread_start_us":89,"threads_started":1}
I20260809 04:49:08.873760 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.866590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42832 (local address 127.7.171.2:37467)
0809 04:49:08.867038 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:08.867052 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:08.867542 (+ 490us) server_negotiation.cc:407] Connection header received
0809 04:49:08.867629 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:08.867634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:08.867804 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:08.867936 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:08.868293 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.869471 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.871875 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:08.872876 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:08.872983 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:08.873152 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:08.873493 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 04:49:08.873601 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":140,"threads_started":1}
I20260809 04:49:08.875296 27720 tablet_service.cc:1467] Processing CreateTablet for tablet 1e07b5e64e58443e9306a59633a4e077 (DEFAULT_TABLE table=test-table [id=58285f6809a14f9a8d29be8d0a54eb3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:08.875385 28006 tablet_service.cc:1467] Processing CreateTablet for tablet 1e07b5e64e58443e9306a59633a4e077 (DEFAULT_TABLE table=test-table [id=58285f6809a14f9a8d29be8d0a54eb3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:08.876480 27720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e07b5e64e58443e9306a59633a4e077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:08.876487 28006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e07b5e64e58443e9306a59633a4e077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:08.877765 27861 tablet_service.cc:1467] Processing CreateTablet for tablet 1e07b5e64e58443e9306a59633a4e077 (DEFAULT_TABLE table=test-table [id=58285f6809a14f9a8d29be8d0a54eb3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:08.878842 27861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e07b5e64e58443e9306a59633a4e077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:08.882633 28096 tablet_bootstrap.cc:492] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8: Bootstrap starting.
I20260809 04:49:08.884158 28097 tablet_bootstrap.cc:492] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: Bootstrap starting.
I20260809 04:49:08.885128 28096 tablet_bootstrap.cc:654] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:08.886011 28096 log.cc:824] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:08.886376 28097 tablet_bootstrap.cc:654] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:08.886512 28098 tablet_bootstrap.cc:492] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e: Bootstrap starting.
I20260809 04:49:08.887264 28097 log.cc:824] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:08.888255 28096 tablet_bootstrap.cc:492] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8: No bootstrap required, opened a new log
I20260809 04:49:08.888463 28096 ts_tablet_manager.cc:1397] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:49:08.889004 28098 tablet_bootstrap.cc:654] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:08.889175 28097 tablet_bootstrap.cc:492] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: No bootstrap required, opened a new log
I20260809 04:49:08.889427 28097 ts_tablet_manager.cc:1397] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20260809 04:49:08.889892 28098 log.cc:824] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:08.891938 28098 tablet_bootstrap.cc:492] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e: No bootstrap required, opened a new log
I20260809 04:49:08.892163 28098 ts_tablet_manager.cc:1397] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:49:08.892967 28097 raft_consensus.cc:348] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42209 } } peers { permanent_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34789 } } peers { permanent_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37467 } }
I20260809 04:49:08.893029 28096 raft_consensus.cc:348] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42209 } } peers { permanent_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34789 } } peers { permanent_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37467 } }
I20260809 04:49:08.893227 28097 raft_consensus.cc:374] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:08.893285 28097 raft_consensus.cc:729] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dbe08859bcf4ac5861d37d2b1fcb151, State: Initialized, Role: FOLLOWER
I20260809 04:49:08.893319 28096 raft_consensus.cc:374] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:08.893415 28096 raft_consensus.cc:729] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa28cd52a8c9481caf518b9cc62a4ed8, State: Initialized, Role: FOLLOWER
I20260809 04:49:08.893827 28097 consensus_queue.cc:260] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42209 } } peers { permanent_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34789 } } peers { permanent_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37467 } }
I20260809 04:49:08.893854 28096 consensus_queue.cc:260] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42209 } } peers { permanent_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34789 } } peers { permanent_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37467 } }
I20260809 04:49:08.894500 28075 heartbeater.cc:503] Master 127.7.171.62:35345 was elected leader, sending a full tablet report...
I20260809 04:49:08.894845 28097 ts_tablet_manager.cc:1428] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260809 04:49:08.894845 28096 ts_tablet_manager.cc:1428] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:49:08.895507 27789 heartbeater.cc:503] Master 127.7.171.62:35345 was elected leader, sending a full tablet report...
I20260809 04:49:08.896471 28098 raft_consensus.cc:348] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42209 } } peers { permanent_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34789 } } peers { permanent_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37467 } }
I20260809 04:49:08.896723 28098 raft_consensus.cc:374] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:08.896812 28098 raft_consensus.cc:729] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf9342ca6f2453bb4d6ea564bbdda4e, State: Initialized, Role: FOLLOWER
I20260809 04:49:08.897270 28098 consensus_queue.cc:260] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42209 } } peers { permanent_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34789 } } peers { permanent_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37467 } }
I20260809 04:49:08.897907 27931 heartbeater.cc:503] Master 127.7.171.62:35345 was elected leader, sending a full tablet report...
I20260809 04:49:08.898276 28098 ts_tablet_manager.cc:1428] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260809 04:49:08.940357 27790 tablet.cc:2369] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:49:09.037616 28102 raft_consensus.cc:482] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:49:09.037843 28102 raft_consensus.cc:504] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42209 } } peers { permanent_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34789 } } peers { permanent_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37467 } }
I20260809 04:49:09.039136 28102 leader_election.cc:288] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa28cd52a8c9481caf518b9cc62a4ed8 (127.7.171.1:34789), bdf9342ca6f2453bb4d6ea564bbdda4e (127.7.171.2:37467)
I20260809 04:49:09.043921 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:09.039543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:45451 (local address 127.7.171.2:37467)
0809 04:49:09.039706 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:09.039713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:09.039756 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:49:09.039822 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:09.039825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:09.039882 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:09.039977 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:09.040403 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:09.041184 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:09.042865 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:09.043331 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:09.043449 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:09.043514 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:09.043711 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 04:49:09.043790 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:49:09.043920 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:09.039403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:37467 (local address 127.7.171.3:45451)
0809 04:49:09.039539 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:09.039554 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:09.039677 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:09.040084 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:09.040089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:09.040119 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:09.040272 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:09.040280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:09.041311 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:09.041314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:09.042716 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:09.042727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:09.043464 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:09.043468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:09.043576 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:09.043598 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:49:09.043665 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 04:49:09.043747 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":34}
I20260809 04:49:09.043956 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:09.039424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:34789 (local address 127.7.171.3:41693)
0809 04:49:09.040068 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:09.040083 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:09.040204 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:09.040511 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:09.040513 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:09.040532 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:09.040645 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:09.040650 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:09.041535 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:09.041538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:09.042735 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:09.042741 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:09.043499 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:09.043511 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:09.043622 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:09.043639 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 04:49:09.043741 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 04:49:09.043803 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":173,"thread_start_us":156,"threads_started":1}
I20260809 04:49:09.043973 28095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:09.039538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:41693 (local address 127.7.171.1:34789)
0809 04:49:09.039685 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:09.039690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:09.040192 (+ 502us) server_negotiation.cc:407] Connection header received
0809 04:49:09.040305 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:09.040307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:09.040379 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:09.040463 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:09.040774 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:09.041426 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:09.042895 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:09.043345 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:09.043481 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:09.043543 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:09.043777 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 04:49:09.043862 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:49:09.044741 27740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e07b5e64e58443e9306a59633a4e077" candidate_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" is_pre_election: true
I20260809 04:49:09.044833 27882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e07b5e64e58443e9306a59633a4e077" candidate_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" is_pre_election: true
I20260809 04:49:09.045096 27740 raft_consensus.cc:2454] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dbe08859bcf4ac5861d37d2b1fcb151 in term 0.
I20260809 04:49:09.045127 27882 raft_consensus.cc:2454] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3dbe08859bcf4ac5861d37d2b1fcb151 in term 0.
I20260809 04:49:09.045583 27953 leader_election.cc:302] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dbe08859bcf4ac5861d37d2b1fcb151, bdf9342ca6f2453bb4d6ea564bbdda4e; no voters:
I20260809 04:49:09.045866 28102 raft_consensus.cc:2781] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:49:09.045961 28102 raft_consensus.cc:482] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:49:09.046007 28102 raft_consensus.cc:3037] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:09.047128 28102 raft_consensus.cc:504] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42209 } } peers { permanent_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34789 } } peers { permanent_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37467 } }
I20260809 04:49:09.047503 28102 leader_election.cc:288] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [CANDIDATE]: Term 1 election: Requested vote from peers aa28cd52a8c9481caf518b9cc62a4ed8 (127.7.171.1:34789), bdf9342ca6f2453bb4d6ea564bbdda4e (127.7.171.2:37467)
I20260809 04:49:09.047825 27740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e07b5e64e58443e9306a59633a4e077" candidate_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8"
I20260809 04:49:09.047865 27882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e07b5e64e58443e9306a59633a4e077" candidate_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e"
I20260809 04:49:09.047955 27882 raft_consensus.cc:3037] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:09.047971 27740 raft_consensus.cc:3037] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:09.048996 27882 raft_consensus.cc:2454] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dbe08859bcf4ac5861d37d2b1fcb151 in term 1.
I20260809 04:49:09.049098 27740 raft_consensus.cc:2454] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3dbe08859bcf4ac5861d37d2b1fcb151 in term 1.
I20260809 04:49:09.049327 27953 leader_election.cc:302] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3dbe08859bcf4ac5861d37d2b1fcb151, bdf9342ca6f2453bb4d6ea564bbdda4e; no voters:
I20260809 04:49:09.049515 28102 raft_consensus.cc:2781] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:09.049770 28102 raft_consensus.cc:686] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 1 LEADER]: Becoming Leader. State: Replica: 3dbe08859bcf4ac5861d37d2b1fcb151, State: Running, Role: LEADER
I20260809 04:49:09.050119 28102 consensus_queue.cc:237] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42209 } } peers { permanent_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34789 } } peers { permanent_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37467 } }
I20260809 04:49:09.053406 27588 catalog_manager.cc:5495] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3dbe08859bcf4ac5861d37d2b1fcb151 (127.7.171.3). New cstate: current_term: 1 leader_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dbe08859bcf4ac5861d37d2b1fcb151" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 42209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37467 } health_report { overall_health: UNKNOWN } } }
I20260809 04:49:09.069119 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:09.065344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53202 (local address 127.7.171.3:42209)
0809 04:49:09.065458 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:09.065462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:09.065481 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:49:09.065637 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:09.065639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:09.065689 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:09.065774 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:09.066041 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:09.066526 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:09.067526 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:09.067740 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:09.067833 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:09.068224 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:09.068631 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:09.068734 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:09.068809 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 04:49:09.068855 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 04:49:09.075106 27882 raft_consensus.cc:1261] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e [term 1 FOLLOWER]: Refusing update from remote peer 3dbe08859bcf4ac5861d37d2b1fcb151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:09.075107 27740 raft_consensus.cc:1261] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8 [term 1 FOLLOWER]: Refusing update from remote peer 3dbe08859bcf4ac5861d37d2b1fcb151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:09.075803 28102 consensus_queue.cc:1035] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa28cd52a8c9481caf518b9cc62a4ed8" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 34789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:09.076007 28106 consensus_queue.cc:1035] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf9342ca6f2453bb4d6ea564bbdda4e" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 37467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 04:49:09.079064 28076 tablet.cc:2369] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:49:09.083631 28110 mvcc.cc:205] Tried to move back new op lower bound from 7316483887404494848 to 7316483887311896576. Current Snapshot: MvccSnapshot[applied={T|T < 7316483887404494848}]
I20260809 04:49:09.085968 28111 mvcc.cc:205] Tried to move back new op lower bound from 7316483887404494848 to 7316483887311896576. Current Snapshot: MvccSnapshot[applied={T|T < 7316483887404494848}]
I20260809 04:49:09.093724 27588 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:09.093904 27588 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:09.096525 27588 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0e7949bab8547b8a6c63ae36db65ef7: Sending DeleteTablet for 3 replicas of tablet 1e07b5e64e58443e9306a59633a4e077
I20260809 04:49:09.097424 28006 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e07b5e64e58443e9306a59633a4e077 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:09 UTC) from {username='slave'} at 127.0.0.1:53186
I20260809 04:49:09.097513 27720 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e07b5e64e58443e9306a59633a4e077 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:09 UTC) from {username='slave'} at 127.0.0.1:52600
I20260809 04:49:09.097640 27861 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e07b5e64e58443e9306a59633a4e077 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:09 UTC) from {username='slave'} at 127.0.0.1:42832
I20260809 04:49:09.097998 28119 tablet_replica.cc:330] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: stopping tablet replica
I20260809 04:49:09.098076 28121 tablet_replica.cc:330] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e: stopping tablet replica
I20260809 04:49:09.098309 28121 raft_consensus.cc:2229] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:49:09.098310 28119 raft_consensus.cc:2229] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:49:09.098486 28120 tablet_replica.cc:330] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8: stopping tablet replica
I20260809 04:49:09.098621 28121 raft_consensus.cc:2258] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:09.098711 28119 raft_consensus.cc:2258] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:09.098775 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 27652
I20260809 04:49:09.099241 28120 raft_consensus.cc:2229] T 1e07b5e64e58443e9306a59633a4e077 P aa28cd52a8c9481caf518b9cc62a4ed8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:49:09.101455 28119 ts_tablet_manager.cc:1905] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:09.104887 28119 ts_tablet_manager.cc:1918] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:09.104982 28119 log.cc:1196] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/1e07b5e64e58443e9306a59633a4e077
I20260809 04:49:09.105266 28119 ts_tablet_manager.cc:1939] T 1e07b5e64e58443e9306a59633a4e077 P 3dbe08859bcf4ac5861d37d2b1fcb151: Deleting consensus metadata
I20260809 04:49:09.106593 27572 catalog_manager.cc:4841] TS 3dbe08859bcf4ac5861d37d2b1fcb151 (127.7.171.3:42209): tablet 1e07b5e64e58443e9306a59633a4e077 (table test-table [id=58285f6809a14f9a8d29be8d0a54eb3a]) successfully deleted
I20260809 04:49:09.109467 28121 ts_tablet_manager.cc:1905] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:09.112987 28121 ts_tablet_manager.cc:1918] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:09.113109 28121 log.cc:1196] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/1e07b5e64e58443e9306a59633a4e077
I20260809 04:49:09.113466 28121 ts_tablet_manager.cc:1939] T 1e07b5e64e58443e9306a59633a4e077 P bdf9342ca6f2453bb4d6ea564bbdda4e: Deleting consensus metadata
W20260809 04:49:09.113824 27574 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:34789 (error 108)
I20260809 04:49:09.114738 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 27792
I20260809 04:49:09.114847 27571 catalog_manager.cc:4841] TS bdf9342ca6f2453bb4d6ea564bbdda4e (127.7.171.2:37467): tablet 1e07b5e64e58443e9306a59633a4e077 (table test-table [id=58285f6809a14f9a8d29be8d0a54eb3a]) successfully deleted
I20260809 04:49:09.117100 28093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:09.116511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:34789 (local address 127.0.0.1:52610)
0809 04:49:09.116741 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:09.116985 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:34789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 04:49:09.117296 27574 catalog_manager.cc:4584] TS aa28cd52a8c9481caf518b9cc62a4ed8 (127.7.171.1:34789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e07b5e64e58443e9306a59633a4e077: Network error: Client connection negotiation failed: client connection to 127.7.171.1:34789: connect: Connection refused (error 111)
I20260809 04:49:09.122032 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 27934
I20260809 04:49:09.129747 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 27556
2026-08-09T04:49:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1152 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:09 dist-test-slave-1zqn krb5kdc[28126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:09 dist-test-slave-1zqn krb5kdc[28126](info): set up 2 sockets
Aug 09 04:49:09 dist-test-slave-1zqn krb5kdc[28126](info): commencing operation
krb5kdc: starting...
W20260809 04:49:10.259963 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:10 dist-test-slave-1zqn krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:49:10.404387 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35493
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:35493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:10.521216 28142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:10.521489 28142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:10.521567 28142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:10.525604 28142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:10.525661 28142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:10.525681 28142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:10.525719 28142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:10.525758 28142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:10.530988 28142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:35493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:10.532198 28142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:10.533303 28142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:10.542454 28150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:10.542488 28147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:10.542454 28148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:10.542804 28142 server_base.cc:1034] running on GCE node
I20260809 04:49:10.543525 28142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:10.544855 28142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:10.546048 28142 hybrid_clock.cc:648] HybridClock initialized: now 1786250950545999 us; error 60 us; skew 500 ppm
Aug 09 04:49:10 dist-test-slave-1zqn krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250950, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:10.549696 28142 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:49:10.550861 28142 webserver.cc:460] Webserver started at http://127.7.171.62:41377/ using document root <none> and password file <none>
I20260809 04:49:10.551519 28142 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:10.551597 28142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:10.551879 28142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:10.554476 28142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "580b1210caae402185c25de9fb6832a1"
format_stamp: "Formatted at 2026-08-09 04:49:10 on dist-test-slave-1zqn"
server_key: "7d9577ace85081c87aebf21f08a2b5af"
server_key_iv: "be84c59d0a55b03719b32da9d4a63b5e"
server_key_version: "encryptionkey@0"
I20260809 04:49:10.555150 28142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "580b1210caae402185c25de9fb6832a1"
format_stamp: "Formatted at 2026-08-09 04:49:10 on dist-test-slave-1zqn"
server_key: "7d9577ace85081c87aebf21f08a2b5af"
server_key_iv: "be84c59d0a55b03719b32da9d4a63b5e"
server_key_version: "encryptionkey@0"
I20260809 04:49:10.559741 28142 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:49:10.562671 28157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:10.563997 28142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:49:10.564205 28142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "580b1210caae402185c25de9fb6832a1"
format_stamp: "Formatted at 2026-08-09 04:49:10 on dist-test-slave-1zqn"
server_key: "7d9577ace85081c87aebf21f08a2b5af"
server_key_iv: "be84c59d0a55b03719b32da9d4a63b5e"
server_key_version: "encryptionkey@0"
I20260809 04:49:10.564337 28142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:10.586876 28142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:10.590487 28142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:10.590710 28142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:10.599181 28142 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:35493
I20260809 04:49:10.599222 28211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:35493 every 8 connection(s)
I20260809 04:49:10.600437 28142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:10.602322 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 28142
I20260809 04:49:10.602504 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:10.602852 7852 external_mini_cluster.cc:1428] Setting key 57bf5d86c27aabe250c1d83522889f85
I20260809 04:49:10.603847 28212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:10.611660 28212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1: Bootstrap starting.
Aug 09 04:49:10 dist-test-slave-1zqn krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:10.614349 28212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:10.615444 28212 log.cc:824] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:10.618166 28212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1: No bootstrap required, opened a new log
I20260809 04:49:10.620000 28215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:10.604674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44798 (local address 127.7.171.62:35493)
0809 04:49:10.605182 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:10.605202 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:10.605248 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:49:10.606174 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:10.606200 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:10.606668 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:10.607091 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:10.607963 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:10.609669 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:10.610678 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:10.611453 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:10.611634 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:10.614340 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:10.614347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:10.614368 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:10.614407 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:10.616518 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:10.617065 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:10.617071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:10.617078 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:10.617249 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:10.617547 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:10.617550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:10.617552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:10.618109 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:10.618320 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:10.618663 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 04:49:10.618989 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":138,"threads_started":1}
I20260809 04:49:10.623042 28212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580b1210caae402185c25de9fb6832a1" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35493 } }
I20260809 04:49:10.623272 28212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:10.623309 28212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580b1210caae402185c25de9fb6832a1, State: Initialized, Role: FOLLOWER
I20260809 04:49:10.623695 28212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580b1210caae402185c25de9fb6832a1" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35493 } }
I20260809 04:49:10.623811 28212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:10.623853 28212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:10.623929 28212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:10.624958 28212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580b1210caae402185c25de9fb6832a1" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35493 } }
I20260809 04:49:10.625278 28212 leader_election.cc:302] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 580b1210caae402185c25de9fb6832a1; no voters:
I20260809 04:49:10.625567 28212 leader_election.cc:288] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:10.625732 28218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:10.626061 28218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [term 1 LEADER]: Becoming Leader. State: Replica: 580b1210caae402185c25de9fb6832a1, State: Running, Role: LEADER
I20260809 04:49:10.626392 28218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580b1210caae402185c25de9fb6832a1" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35493 } }
I20260809 04:49:10.626997 28212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:10.628191 28219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "580b1210caae402185c25de9fb6832a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580b1210caae402185c25de9fb6832a1" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35493 } } }
I20260809 04:49:10.628342 28219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:10.628180 28220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 580b1210caae402185c25de9fb6832a1. Latest consensus state: current_term: 1 leader_uuid: "580b1210caae402185c25de9fb6832a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580b1210caae402185c25de9fb6832a1" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35493 } } }
I20260809 04:49:10.628455 28220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:10.628777 28230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:10.632035 28230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:10.637903 28230 catalog_manager.cc:1347] Generated new cluster ID: dca81d8c833146dd8118aa77641bd0b5
I20260809 04:49:10.637971 28230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:10.646824 28230 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:10.647819 28230 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:10.654520 28230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1: Generated new TSK 0
I20260809 04:49:10.655279 28230 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:49:10.689096 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35493
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:10.806864 28243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:10.807174 28243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:10.807256 28243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:10.811430 28243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:10.811538 28243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:10.811636 28243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:49:10.817098 28243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:10.818250 28243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:10.819356 28243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:10.830114 28251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:10.830083 28248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:10.830073 28249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:10.830509 28243 server_base.cc:1034] running on GCE node
I20260809 04:49:10.830956 28243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:10.831652 28243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:10.832847 28243 hybrid_clock.cc:648] HybridClock initialized: now 1786250950832827 us; error 29 us; skew 500 ppm
Aug 09 04:49:10 dist-test-slave-1zqn krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250950, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:10.835799 28243 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:49:10.836728 28243 webserver.cc:460] Webserver started at http://127.7.171.1:37489/ using document root <none> and password file <none>
I20260809 04:49:10.837270 28243 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:10.837329 28243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:10.837497 28243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:10.839598 28243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "d3f9ad9c242842998300d2be0a72f55c"
format_stamp: "Formatted at 2026-08-09 04:49:10 on dist-test-slave-1zqn"
server_key: "c4828f2540676a67e148d6f601abe7d2"
server_key_iv: "561daa48f15a8502736cf2cbca0ad916"
server_key_version: "encryptionkey@0"
I20260809 04:49:10.840198 28243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "d3f9ad9c242842998300d2be0a72f55c"
format_stamp: "Formatted at 2026-08-09 04:49:10 on dist-test-slave-1zqn"
server_key: "c4828f2540676a67e148d6f601abe7d2"
server_key_iv: "561daa48f15a8502736cf2cbca0ad916"
server_key_version: "encryptionkey@0"
I20260809 04:49:10.843904 28243 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:10.846611 28259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:10.848315 28243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:10.848431 28243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "d3f9ad9c242842998300d2be0a72f55c"
format_stamp: "Formatted at 2026-08-09 04:49:10 on dist-test-slave-1zqn"
server_key: "c4828f2540676a67e148d6f601abe7d2"
server_key_iv: "561daa48f15a8502736cf2cbca0ad916"
server_key_version: "encryptionkey@0"
I20260809 04:49:10.848534 28243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:10.862159 28243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:10.865996 28243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:10.866183 28243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:10.866864 28243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:10.867926 28243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:10.867980 28243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:10.868022 28243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:10.868040 28243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:10.878464 28243 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:46613
I20260809 04:49:10.878494 28385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:46613 every 8 connection(s)
I20260809 04:49:10.879492 28243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:49:10.887347 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 28243
I20260809 04:49:10.887537 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:49:10.887877 7852 external_mini_cluster.cc:1428] Setting key eea8a50f6a4d404dcb62fcdc2b81cdf8
Aug 09 04:49:10 dist-test-slave-1zqn krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250950, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:10.894938 28215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:10.881494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:50011 (local address 127.7.171.62:35493)
0809 04:49:10.881648 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:10.881652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:10.882423 (+ 771us) server_negotiation.cc:407] Connection header received
0809 04:49:10.883796 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:10.883807 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:10.883870 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:10.883984 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:10.885274 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:10.885817 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:10.887007 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:10.887279 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:10.887396 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:10.890368 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:10.890375 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:10.890380 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:10.890415 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:10.892226 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:10.893046 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:10.893049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:10.893050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:10.893141 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:10.893628 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:10.893631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:10.893633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:10.893844 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:10.893940 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:10.894662 (+ 722us) server_negotiation.cc:299] Negotiation successful
0809 04:49:10.894799 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:49:10.896342 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:10.881783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35493 (local address 127.7.171.1:50011)
0809 04:49:10.882264 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:10.882308 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:10.883571 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:10.884234 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:10.884242 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:10.884624 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:10.885118 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:10.885131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:10.885949 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:10.885952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:10.886868 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:10.886877 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:10.887451 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:10.887455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:10.887671 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:10.888401 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:10.888425 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:10.890173 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:10.892353 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:10.892359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:10.892379 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:10.892832 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:10.893247 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:10.893252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:10.893256 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:10.893475 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:10.893973 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:10.893983 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:10.894347 (+ 364us) client_negotiation.cc:769] Sending connection context
0809 04:49:10.894590 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 04:49:10.895026 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":119,"threads_started":1}
I20260809 04:49:10.897938 28386 heartbeater.cc:347] Connected to a master server at 127.7.171.62:35493
I20260809 04:49:10.898332 28386 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:10.898926 28386 heartbeater.cc:511] Master 127.7.171.62:35493 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:49:10.900676 28173 ts_manager.cc:194] Registered new tserver with Master: d3f9ad9c242842998300d2be0a72f55c (127.7.171.1:46613)
I20260809 04:49:10.901921 28173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:50011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:49:10.920419 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35493
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:11.039590 28391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:11.039849 28391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:11.039892 28391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:11.044409 28391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:11.044512 28391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:11.044607 28391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:49:11.050210 28391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:11.051314 28391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:11.052673 28391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:11.063169 28399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:11.063235 28396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:11.063333 28391 server_base.cc:1034] running on GCE node
W20260809 04:49:11.063200 28397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:11.063923 28391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:11.064642 28391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:11.065835 28391 hybrid_clock.cc:648] HybridClock initialized: now 1786250951065800 us; error 47 us; skew 500 ppm
Aug 09 04:49:11 dist-test-slave-1zqn krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250951, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:11.068850 28391 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:49:11.069856 28391 webserver.cc:460] Webserver started at http://127.7.171.2:38933/ using document root <none> and password file <none>
I20260809 04:49:11.070466 28391 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:11.070562 28391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:11.070824 28391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:11.072980 28391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "8a9eb106d06743feb78f4ef49ea6038a"
format_stamp: "Formatted at 2026-08-09 04:49:11 on dist-test-slave-1zqn"
server_key: "a3bf546908cbb97a4407c70c549abe6c"
server_key_iv: "b626eb1b1c301ff1fc8d9793db7b9099"
server_key_version: "encryptionkey@0"
I20260809 04:49:11.073546 28391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "8a9eb106d06743feb78f4ef49ea6038a"
format_stamp: "Formatted at 2026-08-09 04:49:11 on dist-test-slave-1zqn"
server_key: "a3bf546908cbb97a4407c70c549abe6c"
server_key_iv: "b626eb1b1c301ff1fc8d9793db7b9099"
server_key_version: "encryptionkey@0"
I20260809 04:49:11.077519 28391 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:49:11.080250 28406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:11.081570 28391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:49:11.081735 28391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "8a9eb106d06743feb78f4ef49ea6038a"
format_stamp: "Formatted at 2026-08-09 04:49:11 on dist-test-slave-1zqn"
server_key: "a3bf546908cbb97a4407c70c549abe6c"
server_key_iv: "b626eb1b1c301ff1fc8d9793db7b9099"
server_key_version: "encryptionkey@0"
I20260809 04:49:11.081863 28391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:11.098815 28391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:11.102471 28391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:11.102691 28391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:11.103322 28391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:11.104259 28391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:11.104328 28391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:11.104401 28391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:11.104460 28391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:11.113948 28391 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:40427
I20260809 04:49:11.113976 28528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:40427 every 8 connection(s)
I20260809 04:49:11.114841 28391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:49:11.118315 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 28391
I20260809 04:49:11.118475 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:49:11.118785 7852 external_mini_cluster.cc:1428] Setting key 89957e4322e193506e2ded267eb09446
Aug 09 04:49:11 dist-test-slave-1zqn krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250951, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
I20260809 04:49:11.132419 28215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.116761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:37751 (local address 127.7.171.62:35493)
0809 04:49:11.116914 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.116919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.117772 (+ 853us) server_negotiation.cc:407] Connection header received
0809 04:49:11.119373 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.119376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.119451 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.119548 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:11.121385 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.122230 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.124239 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.124525 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.124643 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.127407 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:11.127413 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:11.127419 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:11.127457 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:11.129637 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:11.130235 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:11.130240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:11.130242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:11.130345 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:11.130871 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:11.130878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:11.130881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:11.131127 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:11.131304 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.132035 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.132215 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:49:11.133330 28532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.117100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35493 (local address 127.7.171.2:37751)
0809 04:49:11.117593 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:11.117638 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:11.119128 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:11.119756 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:11.119766 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:11.120444 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:11.121144 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.121179 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.122390 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.122393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.124100 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.124114 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.124705 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.124708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.124926 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.125583 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:11.125600 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:11.127217 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:11.129784 (+ 2567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:11.129790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:11.129803 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:11.130112 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:11.130510 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:11.130514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:11.130517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:11.130719 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:11.131448 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:11.131464 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:11.131738 (+ 274us) client_negotiation.cc:769] Sending connection context
0809 04:49:11.131929 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 04:49:11.132377 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
I20260809 04:49:11.134773 28529 heartbeater.cc:347] Connected to a master server at 127.7.171.62:35493
I20260809 04:49:11.135072 28529 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:11.135617 28529 heartbeater.cc:511] Master 127.7.171.62:35493 requested a full tablet report, sending...
I20260809 04:49:11.136914 28173 ts_manager.cc:194] Registered new tserver with Master: 8a9eb106d06743feb78f4ef49ea6038a (127.7.171.2:40427)
I20260809 04:49:11.137792 28173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:37751
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:49:11.153784 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35493
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:11.269011 28535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:11.269311 28535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:11.269387 28535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:11.273464 28535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:11.273564 28535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:11.273661 28535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:49:11.279035 28535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:35493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:11.280256 28535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:11.281354 28535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:11.291730 28542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:11.291841 28535 server_base.cc:1034] running on GCE node
W20260809 04:49:11.291721 28540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:11.291721 28544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:11.292593 28535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:11.293304 28535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:11.294492 28535 hybrid_clock.cc:648] HybridClock initialized: now 1786250951294473 us; error 33 us; skew 500 ppm
Aug 09 04:49:11 dist-test-slave-1zqn krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250951, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:11.297607 28535 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:49:11.298493 28535 webserver.cc:460] Webserver started at http://127.7.171.3:42081/ using document root <none> and password file <none>
I20260809 04:49:11.299165 28535 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:11.299269 28535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:11.299513 28535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:11.301785 28535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "4aabc87cc3494539a0b737ee576e9437"
format_stamp: "Formatted at 2026-08-09 04:49:11 on dist-test-slave-1zqn"
server_key: "0b51dc964aaefc7ddd75afb58131a771"
server_key_iv: "e328b69a9daf069b47ba619a35086237"
server_key_version: "encryptionkey@0"
I20260809 04:49:11.302376 28535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "4aabc87cc3494539a0b737ee576e9437"
format_stamp: "Formatted at 2026-08-09 04:49:11 on dist-test-slave-1zqn"
server_key: "0b51dc964aaefc7ddd75afb58131a771"
server_key_iv: "e328b69a9daf069b47ba619a35086237"
server_key_version: "encryptionkey@0"
I20260809 04:49:11.306339 28535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:49:11.309101 28551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:11.310521 28535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 04:49:11.310665 28535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "4aabc87cc3494539a0b737ee576e9437"
format_stamp: "Formatted at 2026-08-09 04:49:11 on dist-test-slave-1zqn"
server_key: "0b51dc964aaefc7ddd75afb58131a771"
server_key_iv: "e328b69a9daf069b47ba619a35086237"
server_key_version: "encryptionkey@0"
I20260809 04:49:11.310789 28535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:11.329159 28535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:11.332702 28535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:11.332927 28535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:11.333556 28535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:11.334441 28535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:11.334491 28535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:11.334532 28535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:11.334573 28535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:11.343847 28535 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:36341
I20260809 04:49:11.343887 28670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:36341 every 8 connection(s)
I20260809 04:49:11.344837 28535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:49:11.352178 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 28535
I20260809 04:49:11.352337 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:49:11.352707 7852 external_mini_cluster.cc:1428] Setting key 217bf6bc6084d657f75f859fab1b8d5b
Aug 09 04:49:11 dist-test-slave-1zqn krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250951, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:11.359320 28215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.346980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:34651 (local address 127.7.171.62:35493)
0809 04:49:11.347151 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.347155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.348197 (+ 1042us) server_negotiation.cc:407] Connection header received
0809 04:49:11.349211 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.349214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.349275 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.349383 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:11.350617 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.351153 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.352529 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.352854 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.352975 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.355569 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:11.355572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:11.355575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:11.355601 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:11.357188 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:11.357719 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:11.357722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:11.357723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:11.357802 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:11.358210 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:11.358212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:11.358214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:11.358409 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:11.358493 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.359076 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.359190 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:49:11.360244 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.347389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35493 (local address 127.7.171.3:34651)
0809 04:49:11.347893 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:11.348038 (+ 145us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:11.348989 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:11.349589 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:11.349596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:11.349974 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:11.350468 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.350482 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.351296 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.351300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.352407 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.352421 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.353023 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.353028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.353233 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.353862 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:11.353879 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:11.355373 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:11.357311 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:11.357316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:11.357330 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:11.357625 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:11.357910 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:11.357914 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:11.357916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:11.358094 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:11.358530 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:11.358535 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:11.358821 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 04:49:11.359001 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 04:49:11.359289 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":134,"threads_started":1}
I20260809 04:49:11.361563 28672 heartbeater.cc:347] Connected to a master server at 127.7.171.62:35493
I20260809 04:49:11.361819 28672 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:11.362475 28672 heartbeater.cc:511] Master 127.7.171.62:35493 requested a full tablet report, sending...
I20260809 04:49:11.363557 28173 ts_manager.cc:194] Registered new tserver with Master: 4aabc87cc3494539a0b737ee576e9437 (127.7.171.3:36341)
I20260809 04:49:11.364297 28173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:34651
I20260809 04:49:11.366995 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:49:11.378978 28215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.369416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.7.171.62:35493)
0809 04:49:11.369593 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.369598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.369770 (+ 172us) server_negotiation.cc:407] Connection header received
0809 04:49:11.369939 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.369943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.369995 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.370088 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:11.370868 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.371460 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.372894 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.373144 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.373268 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.375339 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:11.375344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:11.375347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:11.375374 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:11.376867 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:11.377462 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:11.377466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:11.377468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:11.377587 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:11.377959 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:11.377962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:11.377964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:11.378193 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:11.378388 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.378689 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.378794 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:49:11.382082 28173 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:49:11.385521 28173 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:49:11.402637 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.395048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:36341 (local address 127.0.0.1:34542)
0809 04:49:11.395830 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:11.395885 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:11.396045 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:11.397414 (+ 1369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:11.397418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:11.397438 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:11.397580 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.397588 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.399112 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.399116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.400461 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.400472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.401818 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.401823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.402124 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.402179 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 04:49:11.402298 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 04:49:11.402445 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":93,"thread_start_us":103,"threads_started":1}
I20260809 04:49:11.403090 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.396688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55710 (local address 127.7.171.2:40427)
0809 04:49:11.397084 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.397095 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.397115 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:49:11.397223 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.397239 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.397436 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.397605 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:11.398008 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.399629 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.401199 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.401880 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.401996 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.402165 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.402811 (+ 646us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.402928 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":142,"threads_started":1}
I20260809 04:49:11.403143 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.395255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.7.171.3:36341)
0809 04:49:11.396283 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.396296 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.396319 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:49:11.396517 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.396523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.397209 (+ 686us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.397380 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:11.397713 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.398954 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.400615 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.401664 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.401806 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.401985 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.402874 (+ 889us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.402969 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":912,"thread_start_us":140,"threads_started":1}
I20260809 04:49:11.403247 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.395495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56942 (local address 127.7.171.1:46613)
0809 04:49:11.395935 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.395953 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.396211 (+ 258us) server_negotiation.cc:407] Connection header received
0809 04:49:11.396499 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.396505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.396742 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.396887 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:11.397469 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.398787 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.400750 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.401664 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.401806 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.401985 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.402740 (+ 755us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.403074 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":111,"threads_started":1}
I20260809 04:49:11.402694 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.396291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:40427 (local address 127.0.0.1:55710)
0809 04:49:11.396728 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:11.396742 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:11.396825 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:11.397740 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:11.397743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:11.397764 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:11.397871 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.397877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.399800 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.399803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.401067 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.401076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.402233 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.402238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.402394 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.402417 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:49:11.402492 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 04:49:11.402568 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":82,"thread_start_us":122,"threads_started":1}
I20260809 04:49:11.402655 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.395195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:46613 (local address 127.0.0.1:56942)
0809 04:49:11.396101 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:11.396118 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:11.396287 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:11.397139 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:11.397144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:11.397167 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:11.397330 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.397338 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.398929 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.398933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.400607 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.400619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.402092 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.402100 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.402229 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.402277 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 04:49:11.402367 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 04:49:11.402475 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":121,"thread_start_us":112,"threads_started":1}
I20260809 04:49:11.406443 28311 tablet_service.cc:1467] Processing CreateTablet for tablet d0f1bb33dabe49229e7c18aa79644497 (DEFAULT_TABLE table=test-table [id=4c14f8fd5ec64751a1472a2871af9dda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:11.406602 28458 tablet_service.cc:1467] Processing CreateTablet for tablet d0f1bb33dabe49229e7c18aa79644497 (DEFAULT_TABLE table=test-table [id=4c14f8fd5ec64751a1472a2871af9dda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:11.406599 28603 tablet_service.cc:1467] Processing CreateTablet for tablet d0f1bb33dabe49229e7c18aa79644497 (DEFAULT_TABLE table=test-table [id=4c14f8fd5ec64751a1472a2871af9dda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:11.407603 28311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0f1bb33dabe49229e7c18aa79644497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:11.407737 28458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0f1bb33dabe49229e7c18aa79644497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:11.407737 28603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0f1bb33dabe49229e7c18aa79644497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:11.414353 28692 tablet_bootstrap.cc:492] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c: Bootstrap starting.
I20260809 04:49:11.416285 28694 tablet_bootstrap.cc:492] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437: Bootstrap starting.
I20260809 04:49:11.416285 28693 tablet_bootstrap.cc:492] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a: Bootstrap starting.
I20260809 04:49:11.417443 28692 tablet_bootstrap.cc:654] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:11.418315 28692 log.cc:824] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:11.418803 28694 tablet_bootstrap.cc:654] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:11.418803 28693 tablet_bootstrap.cc:654] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:11.420322 28693 log.cc:824] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:11.420322 28694 log.cc:824] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:11.420411 28692 tablet_bootstrap.cc:492] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c: No bootstrap required, opened a new log
I20260809 04:49:11.420644 28692 ts_tablet_manager.cc:1397] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:49:11.422501 28694 tablet_bootstrap.cc:492] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437: No bootstrap required, opened a new log
I20260809 04:49:11.422637 28693 tablet_bootstrap.cc:492] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a: No bootstrap required, opened a new log
I20260809 04:49:11.422761 28694 ts_tablet_manager.cc:1397] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260809 04:49:11.422864 28693 ts_tablet_manager.cc:1397] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 04:49:11.425779 28692 raft_consensus.cc:348] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } }
I20260809 04:49:11.426100 28692 raft_consensus.cc:374] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:11.426167 28692 raft_consensus.cc:729] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3f9ad9c242842998300d2be0a72f55c, State: Initialized, Role: FOLLOWER
I20260809 04:49:11.426688 28692 consensus_queue.cc:260] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } }
I20260809 04:49:11.426749 28694 raft_consensus.cc:348] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } }
I20260809 04:49:11.427150 28694 raft_consensus.cc:374] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:11.427230 28694 raft_consensus.cc:729] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aabc87cc3494539a0b737ee576e9437, State: Initialized, Role: FOLLOWER
I20260809 04:49:11.427327 28693 raft_consensus.cc:348] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } }
I20260809 04:49:11.427645 28693 raft_consensus.cc:374] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:11.427711 28693 raft_consensus.cc:729] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9eb106d06743feb78f4ef49ea6038a, State: Initialized, Role: FOLLOWER
I20260809 04:49:11.427723 28694 consensus_queue.cc:260] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } }
I20260809 04:49:11.427881 28386 heartbeater.cc:503] Master 127.7.171.62:35493 was elected leader, sending a full tablet report...
I20260809 04:49:11.428124 28692 ts_tablet_manager.cc:1428] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 04:49:11.428234 28693 consensus_queue.cc:260] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } }
I20260809 04:49:11.428819 28672 heartbeater.cc:503] Master 127.7.171.62:35493 was elected leader, sending a full tablet report...
I20260809 04:49:11.428841 28694 ts_tablet_manager.cc:1428] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260809 04:49:11.429342 28693 ts_tablet_manager.cc:1428] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260809 04:49:11.429677 28529 heartbeater.cc:503] Master 127.7.171.62:35493 was elected leader, sending a full tablet report...
I20260809 04:49:11.463200 28698 raft_consensus.cc:482] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:49:11.463464 28698 raft_consensus.cc:504] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } }
I20260809 04:49:11.464874 28698 leader_election.cc:288] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4aabc87cc3494539a0b737ee576e9437 (127.7.171.3:36341), 8a9eb106d06743feb78f4ef49ea6038a (127.7.171.2:40427)
I20260809 04:49:11.465579 28700 raft_consensus.cc:482] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:49:11.465790 28700 raft_consensus.cc:504] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } }
I20260809 04:49:11.467291 28700 leader_election.cc:288] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4aabc87cc3494539a0b737ee576e9437 (127.7.171.3:36341), d3f9ad9c242842998300d2be0a72f55c (127.7.171.1:46613)
I20260809 04:49:11.470636 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.465270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:60475 (local address 127.7.171.3:36341)
0809 04:49:11.465748 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.465753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.465774 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:49:11.466016 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.466020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.466125 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.466225 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:11.466652 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.467380 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.469334 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.469940 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.470008 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.470072 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.470427 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.470505 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375}
I20260809 04:49:11.470824 28691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.465505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:45379 (local address 127.7.171.2:40427)
0809 04:49:11.465634 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.465638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.465875 (+ 237us) server_negotiation.cc:407] Connection header received
0809 04:49:11.466020 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.466025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.466152 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.466292 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:11.466720 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.467362 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.469434 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.470066 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.470150 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.470213 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.470586 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.470672 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:49:11.471051 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.465081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:36341 (local address 127.7.171.1:60475)
0809 04:49:11.465531 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:11.465547 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:11.465665 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:11.466278 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:11.466283 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:11.466303 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:11.466475 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.466483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.467547 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.467553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.469199 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.469211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.470168 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.470172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.470268 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.470294 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 04:49:11.470822 (+ 528us) client_negotiation.cc:240] Negotiation successful
0809 04:49:11.470896 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":86,"thread_start_us":151,"threads_started":1}
I20260809 04:49:11.471310 28703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.465307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:40427 (local address 127.7.171.1:45379)
0809 04:49:11.465770 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:11.465784 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:11.465879 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:11.466418 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:11.466421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:11.466439 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:11.466566 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.466590 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.467793 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.467796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.469297 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.469305 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.470315 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.470319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.470432 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.470456 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:49:11.471046 (+ 590us) client_negotiation.cc:240] Negotiation successful
0809 04:49:11.471141 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":2,"thread_start_us":159,"threads_started":1}
I20260809 04:49:11.471711 28623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f1bb33dabe49229e7c18aa79644497" candidate_uuid: "d3f9ad9c242842998300d2be0a72f55c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aabc87cc3494539a0b737ee576e9437" is_pre_election: true
I20260809 04:49:11.472002 28479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f1bb33dabe49229e7c18aa79644497" candidate_uuid: "d3f9ad9c242842998300d2be0a72f55c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9eb106d06743feb78f4ef49ea6038a" is_pre_election: true
I20260809 04:49:11.472081 28623 raft_consensus.cc:2454] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3f9ad9c242842998300d2be0a72f55c in term 0.
I20260809 04:49:11.472257 28479 raft_consensus.cc:2454] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3f9ad9c242842998300d2be0a72f55c in term 0.
I20260809 04:49:11.472589 28264 leader_election.cc:302] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aabc87cc3494539a0b737ee576e9437, d3f9ad9c242842998300d2be0a72f55c; no voters:
I20260809 04:49:11.472945 28698 raft_consensus.cc:2781] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:49:11.473053 28698 raft_consensus.cc:482] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:49:11.473135 28698 raft_consensus.cc:3037] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:11.473307 28532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.467517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:36341 (local address 127.7.171.2:58201)
0809 04:49:11.467674 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:11.467690 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:11.467775 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:11.468693 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:11.468696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:11.468717 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:11.468827 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.468833 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.469888 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.469891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.471075 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.471088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.472871 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.472875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.473025 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.473052 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:49:11.473127 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 04:49:11.473195 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 04:49:11.473791 28707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.468042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:58201 (local address 127.7.171.3:36341)
0809 04:49:11.468438 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.468449 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.468465 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:49:11.468512 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.468515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.468562 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.468658 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:11.468974 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.469763 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.471256 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.472477 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.473489 (+ 1012us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.473558 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.473595 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.473658 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":104,"threads_started":1}
I20260809 04:49:11.474231 28623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f1bb33dabe49229e7c18aa79644497" candidate_uuid: "8a9eb106d06743feb78f4ef49ea6038a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aabc87cc3494539a0b737ee576e9437" is_pre_election: true
I20260809 04:49:11.474372 28623 raft_consensus.cc:2454] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a9eb106d06743feb78f4ef49ea6038a in term 0.
I20260809 04:49:11.474643 28698 raft_consensus.cc:504] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } }
I20260809 04:49:11.474752 28410 leader_election.cc:302] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aabc87cc3494539a0b737ee576e9437, 8a9eb106d06743feb78f4ef49ea6038a; no voters:
I20260809 04:49:11.475078 28698 leader_election.cc:288] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [CANDIDATE]: Term 1 election: Requested vote from peers 4aabc87cc3494539a0b737ee576e9437 (127.7.171.3:36341), 8a9eb106d06743feb78f4ef49ea6038a (127.7.171.2:40427)
I20260809 04:49:11.475069 28700 raft_consensus.cc:2781] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:49:11.475176 28700 raft_consensus.cc:482] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:49:11.475198 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.468011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:51163 (local address 127.7.171.1:46613)
0809 04:49:11.468151 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.468156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.469059 (+ 903us) server_negotiation.cc:407] Connection header received
0809 04:49:11.469248 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.469251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.469298 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.469353 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:11.469822 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.470497 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.473714 (+ 3217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.474338 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.474441 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.474530 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.474947 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.475025 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:49:11.475232 28700 raft_consensus.cc:3037] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:11.475536 28623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f1bb33dabe49229e7c18aa79644497" candidate_uuid: "d3f9ad9c242842998300d2be0a72f55c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aabc87cc3494539a0b737ee576e9437"
I20260809 04:49:11.475510 28479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f1bb33dabe49229e7c18aa79644497" candidate_uuid: "d3f9ad9c242842998300d2be0a72f55c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a9eb106d06743feb78f4ef49ea6038a"
I20260809 04:49:11.475685 28706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.467623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:46613 (local address 127.7.171.2:51163)
0809 04:49:11.468971 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:11.468987 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:11.469122 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:11.469507 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:11.469510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:11.469528 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:11.469688 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.469695 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.470637 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.470640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.472777 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:11.472791 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.474675 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.474681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:11.474782 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.474809 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:49:11.474910 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 04:49:11.474981 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"mutex_wait_us":38,"thread_start_us":114,"threads_started":1}
I20260809 04:49:11.475675 28623 raft_consensus.cc:3037] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:11.476435 28334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f1bb33dabe49229e7c18aa79644497" candidate_uuid: "8a9eb106d06743feb78f4ef49ea6038a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f9ad9c242842998300d2be0a72f55c" is_pre_election: true
I20260809 04:49:11.476622 28700 raft_consensus.cc:504] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } }
I20260809 04:49:11.476717 28334 raft_consensus.cc:2379] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a9eb106d06743feb78f4ef49ea6038a in current term 1: Already voted for candidate d3f9ad9c242842998300d2be0a72f55c in this term.
I20260809 04:49:11.476882 28479 raft_consensus.cc:2379] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d3f9ad9c242842998300d2be0a72f55c in current term 1: Already voted for candidate 8a9eb106d06743feb78f4ef49ea6038a in this term.
I20260809 04:49:11.477000 28700 leader_election.cc:288] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [CANDIDATE]: Term 1 election: Requested vote from peers 4aabc87cc3494539a0b737ee576e9437 (127.7.171.3:36341), d3f9ad9c242842998300d2be0a72f55c (127.7.171.1:46613)
I20260809 04:49:11.477203 28623 raft_consensus.cc:2454] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3f9ad9c242842998300d2be0a72f55c in term 1.
I20260809 04:49:11.477461 28622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f1bb33dabe49229e7c18aa79644497" candidate_uuid: "8a9eb106d06743feb78f4ef49ea6038a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aabc87cc3494539a0b737ee576e9437"
I20260809 04:49:11.477540 28264 leader_election.cc:302] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4aabc87cc3494539a0b737ee576e9437, d3f9ad9c242842998300d2be0a72f55c; no voters:
I20260809 04:49:11.477620 28622 raft_consensus.cc:2379] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a9eb106d06743feb78f4ef49ea6038a in current term 1: Already voted for candidate d3f9ad9c242842998300d2be0a72f55c in this term.
I20260809 04:49:11.477717 28698 raft_consensus.cc:2781] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:11.477970 28698 raft_consensus.cc:686] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [term 1 LEADER]: Becoming Leader. State: Replica: d3f9ad9c242842998300d2be0a72f55c, State: Running, Role: LEADER
I20260809 04:49:11.478107 28334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0f1bb33dabe49229e7c18aa79644497" candidate_uuid: "8a9eb106d06743feb78f4ef49ea6038a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3f9ad9c242842998300d2be0a72f55c"
I20260809 04:49:11.478341 28698 consensus_queue.cc:237] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } }
I20260809 04:49:11.478488 28408 leader_election.cc:302] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a9eb106d06743feb78f4ef49ea6038a; no voters: 4aabc87cc3494539a0b737ee576e9437, d3f9ad9c242842998300d2be0a72f55c
I20260809 04:49:11.478657 28700 raft_consensus.cc:2726] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 04:49:11.481808 28174 catalog_manager.cc:5495] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c reported cstate change: term changed from 0 to 1, leader changed from <none> to d3f9ad9c242842998300d2be0a72f55c (127.7.171.1). New cstate: current_term: 1 leader_uuid: "d3f9ad9c242842998300d2be0a72f55c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3f9ad9c242842998300d2be0a72f55c" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 46613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 } health_report { overall_health: UNKNOWN } } }
I20260809 04:49:11.502630 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.498682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56952 (local address 127.7.171.1:46613)
0809 04:49:11.498865 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.498869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.498886 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:49:11.498974 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.498977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.499055 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.499163 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:11.499425 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.499960 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.500905 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.501160 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.501255 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.501719 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:11.502174 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:11.502311 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.502455 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.502492 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 04:49:11.508991 28479 raft_consensus.cc:1261] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 1 FOLLOWER]: Refusing update from remote peer d3f9ad9c242842998300d2be0a72f55c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:11.508993 28622 raft_consensus.cc:1261] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [term 1 FOLLOWER]: Refusing update from remote peer d3f9ad9c242842998300d2be0a72f55c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:11.509663 28698 consensus_queue.cc:1035] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aabc87cc3494539a0b737ee576e9437" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 36341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:11.509882 28708 consensus_queue.cc:1035] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a9eb106d06743feb78f4ef49ea6038a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 40427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:11.517320 28711 mvcc.cc:205] Tried to move back new op lower bound from 7316483897373323264 to 7316483897258536960. Current Snapshot: MvccSnapshot[applied={T|T < 7316483897373323264}]
I20260809 04:49:11.518976 28713 mvcc.cc:205] Tried to move back new op lower bound from 7316483897373323264 to 7316483897258536960. Current Snapshot: MvccSnapshot[applied={T|T < 7316483897373323264}]
I20260809 04:49:11.528226 28174 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:11.528453 28174 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:11.531155 28174 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 580b1210caae402185c25de9fb6832a1: Sending DeleteTablet for 3 replicas of tablet d0f1bb33dabe49229e7c18aa79644497
I20260809 04:49:11.532032 28311 tablet_service.cc:1514] Processing DeleteTablet for tablet d0f1bb33dabe49229e7c18aa79644497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:11 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:56942
I20260809 04:49:11.532032 28603 tablet_service.cc:1514] Processing DeleteTablet for tablet d0f1bb33dabe49229e7c18aa79644497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:11 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:34542
I20260809 04:49:11.532022 28458 tablet_service.cc:1514] Processing DeleteTablet for tablet d0f1bb33dabe49229e7c18aa79644497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:11 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:55710
I20260809 04:49:11.532629 28724 tablet_replica.cc:330] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a: stopping tablet replica
I20260809 04:49:11.532670 28723 tablet_replica.cc:330] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437: stopping tablet replica
I20260809 04:49:11.533051 28724 raft_consensus.cc:2229] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:49:11.533052 28723 raft_consensus.cc:2229] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:49:11.533402 28723 raft_consensus.cc:2258] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:11.533434 28724 raft_consensus.cc:2258] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:11.533622 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 28243
I20260809 04:49:11.533805 28722 tablet_replica.cc:330] T d0f1bb33dabe49229e7c18aa79644497 P d3f9ad9c242842998300d2be0a72f55c: stopping tablet replica
I20260809 04:49:11.537374 28723 ts_tablet_manager.cc:1905] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:11.540367 28724 ts_tablet_manager.cc:1905] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:11.541112 28723 ts_tablet_manager.cc:1918] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:11.541289 28723 log.cc:1196] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/d0f1bb33dabe49229e7c18aa79644497
I20260809 04:49:11.541738 28723 ts_tablet_manager.cc:1939] T d0f1bb33dabe49229e7c18aa79644497 P 4aabc87cc3494539a0b737ee576e9437: Deleting consensus metadata
I20260809 04:49:11.543200 28724 ts_tablet_manager.cc:1918] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:11.543237 28161 catalog_manager.cc:4841] TS 4aabc87cc3494539a0b737ee576e9437 (127.7.171.3:36341): tablet d0f1bb33dabe49229e7c18aa79644497 (table test-table [id=4c14f8fd5ec64751a1472a2871af9dda]) successfully deleted
I20260809 04:49:11.543315 28724 log.cc:1196] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/d0f1bb33dabe49229e7c18aa79644497
I20260809 04:49:11.543649 28724 ts_tablet_manager.cc:1939] T d0f1bb33dabe49229e7c18aa79644497 P 8a9eb106d06743feb78f4ef49ea6038a: Deleting consensus metadata
I20260809 04:49:11.544822 28158 catalog_manager.cc:4841] TS 8a9eb106d06743feb78f4ef49ea6038a (127.7.171.2:40427): tablet d0f1bb33dabe49229e7c18aa79644497 (table test-table [id=4c14f8fd5ec64751a1472a2871af9dda]) successfully deleted
W20260809 04:49:11.545189 28159 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:46613 (error 108)
I20260809 04:49:11.545457 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 28391
I20260809 04:49:11.547102 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.546801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:46613 (local address 127.0.0.1:56964)
0809 04:49:11.546937 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:11.547012 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 04:49:11.547330 28159 catalog_manager.cc:4584] TS d3f9ad9c242842998300d2be0a72f55c (127.7.171.1:46613): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0f1bb33dabe49229e7c18aa79644497: Network error: Client connection negotiation failed: client connection to 127.7.171.1:46613: connect: Connection refused (error 111)
I20260809 04:49:11.554533 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 28535
I20260809 04:49:11.562266 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 28142
2026-08-09T04:49:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2434 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T04:49:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:11Z Disabled control of system clock
I20260809 04:49:11.599134 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:41527
--webserver_interface=127.7.171.62
--webserver_port=0
--builtin_ntp_servers=127.7.171.20:42409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:41527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:49:11.715870 28730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:11.716156 28730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:11.716192 28730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:11.720098 28730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:11.720160 28730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:11.720175 28730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:11.720192 28730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:11.720207 28730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:11.725134 28730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:41527
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:41527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:11.726150 28730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:11.727478 28730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:11.736048 28739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:11.736115 28735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:11.736048 28736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:11.736248 28730 server_base.cc:1034] running on GCE node
I20260809 04:49:11.736943 28730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:11.737974 28730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:11.739197 28730 hybrid_clock.cc:648] HybridClock initialized: now 1786250951739162 us; error 50 us; skew 500 ppm
I20260809 04:49:11.745545 28730 webserver.cc:460] Webserver started at http://127.7.171.62:42837/ using document root <none> and password file <none>
I20260809 04:49:11.746156 28730 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:11.746222 28730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:11.746404 28730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:11.748696 28730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "2b12d60b882f43ed8c0c785d6847ba7a"
format_stamp: "Formatted at 2026-08-09 04:49:11 on dist-test-slave-1zqn"
server_key: "ac5c2b84c571cbff89ba6264eab0dcc4"
server_key_iv: "5f82b719e59d24b1b6547802ab29a6e3"
server_key_version: "encryptionkey@0"
I20260809 04:49:11.749257 28730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "2b12d60b882f43ed8c0c785d6847ba7a"
format_stamp: "Formatted at 2026-08-09 04:49:11 on dist-test-slave-1zqn"
server_key: "ac5c2b84c571cbff89ba6264eab0dcc4"
server_key_iv: "5f82b719e59d24b1b6547802ab29a6e3"
server_key_version: "encryptionkey@0"
I20260809 04:49:11.752962 28730 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:49:11.755443 28745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:11.756841 28730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:49:11.757030 28730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "2b12d60b882f43ed8c0c785d6847ba7a"
format_stamp: "Formatted at 2026-08-09 04:49:11 on dist-test-slave-1zqn"
server_key: "ac5c2b84c571cbff89ba6264eab0dcc4"
server_key_iv: "5f82b719e59d24b1b6547802ab29a6e3"
server_key_version: "encryptionkey@0"
I20260809 04:49:11.757169 28730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:11.797429 28730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:11.798209 28730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:11.798416 28730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:11.806352 28730 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:41527
I20260809 04:49:11.806375 28801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:41527 every 8 connection(s)
I20260809 04:49:11.807332 28730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:11.810582 28802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:11.816344 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 28730
I20260809 04:49:11.816385 28802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a: Bootstrap starting.
I20260809 04:49:11.816493 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:11.816787 7852 external_mini_cluster.cc:1428] Setting key 867601aeef5be1d5a390484ec09af6ee
I20260809 04:49:11.819020 28802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:11.819772 28802 log.cc:824] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:11.822074 28802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a: No bootstrap required, opened a new log
I20260809 04:49:11.826599 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:11.818214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35950 (local address 127.7.171.62:41527)
0809 04:49:11.818904 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:11.818918 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:11.818952 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:49:11.819624 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:11.819642 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:11.820152 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:11.820550 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:11.821064 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.822023 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.823237 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:11.823690 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:11.823880 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:11.824729 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:11.824734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:11.824746 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:11.825169 (+ 423us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:11.825223 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:11.825243 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:11.825320 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:11.825514 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 04:49:11.825797 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":155,"threads_started":1}
I20260809 04:49:11.826464 28802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b12d60b882f43ed8c0c785d6847ba7a" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41527 } }
I20260809 04:49:11.826768 28802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:11.826834 28802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b12d60b882f43ed8c0c785d6847ba7a, State: Initialized, Role: FOLLOWER
I20260809 04:49:11.827466 28802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b12d60b882f43ed8c0c785d6847ba7a" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41527 } }
I20260809 04:49:11.827646 28802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:11.827729 28802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:11.827845 28802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:11.829180 28802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b12d60b882f43ed8c0c785d6847ba7a" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41527 } }
I20260809 04:49:11.829600 28802 leader_election.cc:302] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b12d60b882f43ed8c0c785d6847ba7a; no voters:
I20260809 04:49:11.829991 28802 leader_election.cc:288] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:11.830142 28808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:11.830435 28808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [term 1 LEADER]: Becoming Leader. State: Replica: 2b12d60b882f43ed8c0c785d6847ba7a, State: Running, Role: LEADER
I20260809 04:49:11.830785 28808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b12d60b882f43ed8c0c785d6847ba7a" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41527 } }
I20260809 04:49:11.831492 28802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:11.832114 28809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b12d60b882f43ed8c0c785d6847ba7a. Latest consensus state: current_term: 1 leader_uuid: "2b12d60b882f43ed8c0c785d6847ba7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b12d60b882f43ed8c0c785d6847ba7a" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41527 } } }
I20260809 04:49:11.832249 28809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:11.832552 28810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b12d60b882f43ed8c0c785d6847ba7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b12d60b882f43ed8c0c785d6847ba7a" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 41527 } } }
I20260809 04:49:11.832652 28810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:11.832772 28812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:11.836128 28812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:11.843118 28812 catalog_manager.cc:1347] Generated new cluster ID: 4d5f757ca8514c9aa482f1604289b9c7
I20260809 04:49:11.843204 28812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:11.866314 28812 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:11.867592 28812 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:11.888868 28812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a: Generated new TSK 0
I20260809 04:49:11.889853 28812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:11.902997 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41527
--builtin_ntp_servers=127.7.171.20:42409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:49:12.026278 28828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:12.026563 28828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:12.026639 28828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:12.031036 28828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:12.031095 28828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:12.031183 28828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:49:12.036290 28828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:12.037402 28828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:12.038848 28828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:12.048614 28838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:12.048671 28828 server_base.cc:1034] running on GCE node
W20260809 04:49:12.048614 28835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:12.048632 28834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:12.049299 28828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:12.049885 28828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:12.051090 28828 hybrid_clock.cc:648] HybridClock initialized: now 1786250952051067 us; error 43 us; skew 500 ppm
I20260809 04:49:12.058476 28828 webserver.cc:460] Webserver started at http://127.7.171.1:44709/ using document root <none> and password file <none>
I20260809 04:49:12.059151 28828 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:12.059257 28828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:12.059494 28828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:12.061893 28828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "9f6dbcd62224458d8fb27a92410f6e05"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1zqn"
server_key: "7f8f923d97b16f32912abc7e5dcefa14"
server_key_iv: "efbca8ccc8ac5c6a9ab61ff8c2c958b8"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.062472 28828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "9f6dbcd62224458d8fb27a92410f6e05"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1zqn"
server_key: "7f8f923d97b16f32912abc7e5dcefa14"
server_key_iv: "efbca8ccc8ac5c6a9ab61ff8c2c958b8"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.066257 28828 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:49:12.068817 28844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:12.070133 28828 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:49:12.070272 28828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "9f6dbcd62224458d8fb27a92410f6e05"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1zqn"
server_key: "7f8f923d97b16f32912abc7e5dcefa14"
server_key_iv: "efbca8ccc8ac5c6a9ab61ff8c2c958b8"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.070411 28828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:12.087980 28828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:12.088837 28828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:12.089064 28828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:12.089699 28828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:12.090689 28828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:12.090763 28828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:12.090843 28828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:12.090888 28828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:12.100584 28828 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:38425
I20260809 04:49:12.100791 28966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:38425 every 8 connection(s)
I20260809 04:49:12.101656 28828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:49:12.111327 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 28828
I20260809 04:49:12.111454 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.103777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:47207 (local address 127.7.171.62:41527)
0809 04:49:12.103919 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.103923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.105007 (+ 1084us) server_negotiation.cc:407] Connection header received
0809 04:49:12.106582 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.106586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.106648 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.106728 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:12.107641 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.108272 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.109460 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.109694 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.109807 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.110914 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:12.110918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:12.110921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:12.110941 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:12.110960 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:12.110968 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:12.111080 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.111241 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.111305 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:49:12.111482 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:49:12.111814 7852 external_mini_cluster.cc:1428] Setting key 55a5b817bd9b4518bb00965477e4d03e
I20260809 04:49:12.112398 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.104173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:41527 (local address 127.7.171.1:47207)
0809 04:49:12.104843 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:12.104883 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:12.106351 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:12.106903 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:12.106911 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:12.106995 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:12.107489 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.107501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.108403 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.108411 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.109331 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.109341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.109851 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.109854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.110020 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.110650 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:49:12.110652 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:12.110691 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:49:12.110753 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:12.111085 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:12.111097 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:12.111105 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:49:12.111239 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 04:49:12.111508 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":124,"threads_started":1}
I20260809 04:49:12.113693 28967 heartbeater.cc:347] Connected to a master server at 127.7.171.62:41527
I20260809 04:49:12.113941 28967 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:12.114068 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41527
--builtin_ntp_servers=127.7.171.20:42409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:49:12.114499 28967 heartbeater.cc:511] Master 127.7.171.62:41527 requested a full tablet report, sending...
I20260809 04:49:12.116189 28762 ts_manager.cc:194] Registered new tserver with Master: 9f6dbcd62224458d8fb27a92410f6e05 (127.7.171.1:38425)
I20260809 04:49:12.117414 28762 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.1:47207
W20260809 04:49:12.228255 28970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:12.228508 28970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:12.228592 28970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:12.232638 28970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:12.232692 28970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:12.232777 28970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:49:12.237494 28970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:12.238615 28970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:12.240036 28970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:12.252027 28979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:12.252280 28970 server_base.cc:1034] running on GCE node
W20260809 04:49:12.252027 28977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:12.252027 28975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:12.252799 28970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:12.253455 28970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:12.254652 28970 hybrid_clock.cc:648] HybridClock initialized: now 1786250952254635 us; error 29 us; skew 500 ppm
I20260809 04:49:12.256563 28970 webserver.cc:460] Webserver started at http://127.7.171.2:32869/ using document root <none> and password file <none>
I20260809 04:49:12.257148 28970 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:12.257241 28970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:12.257472 28970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:12.259549 28970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "633bfef3d9464c4994470a75e11e6c01"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1zqn"
server_key: "382a16c65a5431805752f6083db8e39f"
server_key_iv: "43e286b3d5f74d7551ac1a5a06e93861"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.260210 28970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "633bfef3d9464c4994470a75e11e6c01"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1zqn"
server_key: "382a16c65a5431805752f6083db8e39f"
server_key_iv: "43e286b3d5f74d7551ac1a5a06e93861"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.264288 28970 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:49:12.267135 28986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:12.268563 28970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:12.268697 28970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "633bfef3d9464c4994470a75e11e6c01"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1zqn"
server_key: "382a16c65a5431805752f6083db8e39f"
server_key_iv: "43e286b3d5f74d7551ac1a5a06e93861"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.268808 28970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:12.279198 28970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:12.279876 28970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:12.280164 28970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:12.280759 28970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:12.281704 28970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:12.281777 28970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:12.281852 28970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:12.281893 28970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:12.291529 28970 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:34057
I20260809 04:49:12.291548 29112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:34057 every 8 connection(s)
I20260809 04:49:12.292464 28970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:49:12.302088 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 28970
I20260809 04:49:12.302115 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.294418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:43591 (local address 127.7.171.62:41527)
0809 04:49:12.294602 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.294605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.295733 (+ 1128us) server_negotiation.cc:407] Connection header received
0809 04:49:12.297256 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.297259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.297318 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.297432 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:12.298326 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.298873 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.300036 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.300311 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.300425 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.301591 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:12.301595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:12.301598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:12.301617 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:12.301638 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:12.301646 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:12.301728 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.301892 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.301957 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 04:49:12.302237 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:49:12.302551 7852 external_mini_cluster.cc:1428] Setting key 12003cec707e1baa7d78dc221792c9b5
I20260809 04:49:12.303067 29115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.294828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:41527 (local address 127.7.171.2:43591)
0809 04:49:12.295571 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:12.295613 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:12.297032 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:12.297614 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:12.297621 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:12.297706 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:12.298180 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.298192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.299003 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.299006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.299895 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.299905 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.300468 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.300472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.300643 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.301308 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:49:12.301310 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:12.301349 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:49:12.301423 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:12.301731 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:12.301743 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:12.301752 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:49:12.301890 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 04:49:12.302162 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":120,"threads_started":1}
I20260809 04:49:12.304270 29113 heartbeater.cc:347] Connected to a master server at 127.7.171.62:41527
I20260809 04:49:12.304534 29113 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:12.305011 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41527
--builtin_ntp_servers=127.7.171.20:42409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:49:12.305167 29113 heartbeater.cc:511] Master 127.7.171.62:41527 requested a full tablet report, sending...
I20260809 04:49:12.306358 28762 ts_manager.cc:194] Registered new tserver with Master: 633bfef3d9464c4994470a75e11e6c01 (127.7.171.2:34057)
I20260809 04:49:12.307155 28762 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.2:43591
W20260809 04:49:12.426292 29117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:12.426590 29117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:12.426631 29117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:12.430802 29117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:12.430878 29117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:12.430967 29117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:49:12.436286 29117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:12.437484 29117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:12.439010 29117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:49:12.448900 29117 server_base.cc:1034] running on GCE node
W20260809 04:49:12.448869 29125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:12.449160 29123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:12.449105 29122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:12.449631 29117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:12.450278 29117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:12.451472 29117 hybrid_clock.cc:648] HybridClock initialized: now 1786250952451435 us; error 54 us; skew 500 ppm
I20260809 04:49:12.453372 29117 webserver.cc:460] Webserver started at http://127.7.171.3:38669/ using document root <none> and password file <none>
I20260809 04:49:12.453962 29117 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:12.454066 29117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:12.454305 29117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:12.456445 29117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "801db1f6ffcc4a4798de83c7bc79128c"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1zqn"
server_key: "51b4da6232796d1c66c3f3ffa5acd85f"
server_key_iv: "5f0633579a5bded4bc249ad38d7858f3"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.457026 29117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "801db1f6ffcc4a4798de83c7bc79128c"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1zqn"
server_key: "51b4da6232796d1c66c3f3ffa5acd85f"
server_key_iv: "5f0633579a5bded4bc249ad38d7858f3"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.460927 29117 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:49:12.463493 29134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:12.464767 29117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:12.464905 29117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "801db1f6ffcc4a4798de83c7bc79128c"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1zqn"
server_key: "51b4da6232796d1c66c3f3ffa5acd85f"
server_key_iv: "5f0633579a5bded4bc249ad38d7858f3"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.465023 29117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:12.482271 29117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:12.482990 29117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:12.483209 29117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:12.483826 29117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:12.484839 29117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:12.484913 29117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:12.484985 29117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:12.485034 29117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:12.494822 29117 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:37793
I20260809 04:49:12.494844 29256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:37793 every 8 connection(s)
I20260809 04:49:12.495820 29117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:49:12.504119 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 29117
I20260809 04:49:12.504268 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:49:12.504599 7852 external_mini_cluster.cc:1428] Setting key 7b9ef048185347364ce9d9d58f86f275
I20260809 04:49:12.506464 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.497969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:45421 (local address 127.7.171.62:41527)
0809 04:49:12.498128 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.498133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.499369 (+ 1236us) server_negotiation.cc:407] Connection header received
0809 04:49:12.500852 (+ 1483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.500854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.500902 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.500979 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:12.501971 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.502547 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.503895 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.504237 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.504397 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.505827 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:12.505831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:12.505834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:12.505849 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:12.505880 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:12.505889 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:12.506030 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.506229 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.506299 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:49:12.507501 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.498426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:41527 (local address 127.7.171.3:45421)
0809 04:49:12.499214 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:12.499258 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:12.500646 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:12.501167 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:12.501175 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:12.501266 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:12.501797 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.501811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.502715 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.502718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.503718 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.503728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.504458 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.504464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.504687 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.505466 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:49:12.505469 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:12.505528 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:49:12.505637 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:12.506048 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:12.506061 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:12.506068 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 04:49:12.506222 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 04:49:12.506533 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":132,"threads_started":1}
I20260809 04:49:12.508697 29257 heartbeater.cc:347] Connected to a master server at 127.7.171.62:41527
I20260809 04:49:12.508937 29257 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:12.509478 29257 heartbeater.cc:511] Master 127.7.171.62:41527 requested a full tablet report, sending...
I20260809 04:49:12.510502 28762 ts_manager.cc:194] Registered new tserver with Master: 801db1f6ffcc4a4798de83c7bc79128c (127.7.171.3:37793)
I20260809 04:49:12.511253 28762 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.3:45421
I20260809 04:49:12.518770 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:49:12.524887 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.520868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35962 (local address 127.7.171.62:41527)
0809 04:49:12.521041 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.521046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.521219 (+ 173us) server_negotiation.cc:407] Connection header received
0809 04:49:12.521370 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.521373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.521413 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.521496 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:12.521797 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.522307 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.523161 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.523375 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.523481 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.524382 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:12.524386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:12.524388 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:12.524399 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:12.524421 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:12.524429 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:12.524569 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.524692 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.524742 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 04:49:12.527827 28762 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:49:12.530777 28762 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:49:12.545399 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.539891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.7.171.2:34057)
0809 04:49:12.540335 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.540346 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.540930 (+ 584us) server_negotiation.cc:407] Connection header received
0809 04:49:12.541022 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.541028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.541216 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.541377 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:12.541621 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.542617 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.544191 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.544726 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.544845 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.545054 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.545139 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.545232 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":160,"threads_started":1}
I20260809 04:49:12.545398 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.539690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:34057 (local address 127.0.0.1:41838)
0809 04:49:12.540709 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:12.540724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:12.540819 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:12.541351 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:12.541354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:12.541382 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:12.541480 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.541490 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.542736 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.542740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.544022 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.544031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.544857 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.544861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.544964 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.544999 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 04:49:12.545111 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 04:49:12.545231 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":956,"mutex_wait_us":63,"spinlock_wait_cycles":768,"thread_start_us":93,"threads_started":1}
I20260809 04:49:12.546854 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.540224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:38425 (local address 127.0.0.1:53870)
0809 04:49:12.541572 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:12.541588 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:12.541658 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:12.542107 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:12.542110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:12.542128 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:12.542208 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.542213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.543715 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.543718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.544842 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.544852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.545906 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.545909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.546475 (+ 566us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.546499 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:49:12.546622 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 04:49:12.546691 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1272,"mutex_wait_us":456,"thread_start_us":98,"threads_started":1}
I20260809 04:49:12.547063 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.540672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53870 (local address 127.7.171.1:38425)
0809 04:49:12.541690 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.541703 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.541725 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:49:12.541796 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.541800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.541963 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.542106 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:12.542341 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.543587 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.544986 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.545794 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.545902 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.546071 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.546788 (+ 717us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.546884 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":907,"thread_start_us":145,"threads_started":1}
I20260809 04:49:12.547314 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.539461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:37793 (local address 127.0.0.1:52456)
0809 04:49:12.540450 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:12.540487 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:12.540642 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:12.541304 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:12.541310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:12.541333 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:12.541476 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.541484 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.542760 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.542763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.544149 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.544159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.545457 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.545460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.546429 (+ 969us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.546454 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:49:12.546577 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 04:49:12.546652 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":56,"spinlock_wait_cycles":1280,"thread_start_us":95,"threads_started":1}
I20260809 04:49:12.547132 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.539621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52456 (local address 127.7.171.3:37793)
0809 04:49:12.540191 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.540206 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.540803 (+ 597us) server_negotiation.cc:407] Connection header received
0809 04:49:12.540894 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.540900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.541141 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.541316 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:12.541628 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.542630 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.544286 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.545089 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.545163 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.545346 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.546622 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.546727 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":174,"threads_started":1}
I20260809 04:49:12.548792 29038 tablet_service.cc:1467] Processing CreateTablet for tablet c748fa4482b14797a9fedf6aeea20658 (DEFAULT_TABLE table=test-table [id=bd5b894108c24011af1cc266c95180fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:12.549649 28897 tablet_service.cc:1467] Processing CreateTablet for tablet c748fa4482b14797a9fedf6aeea20658 (DEFAULT_TABLE table=test-table [id=bd5b894108c24011af1cc266c95180fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:12.550042 29038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c748fa4482b14797a9fedf6aeea20658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:12.550541 28897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c748fa4482b14797a9fedf6aeea20658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:12.550465 29186 tablet_service.cc:1467] Processing CreateTablet for tablet c748fa4482b14797a9fedf6aeea20658 (DEFAULT_TABLE table=test-table [id=bd5b894108c24011af1cc266c95180fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:12.551442 29186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c748fa4482b14797a9fedf6aeea20658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:12.557338 29277 tablet_bootstrap.cc:492] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05: Bootstrap starting.
I20260809 04:49:12.558105 29278 tablet_bootstrap.cc:492] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01: Bootstrap starting.
I20260809 04:49:12.558837 29279 tablet_bootstrap.cc:492] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c: Bootstrap starting.
I20260809 04:49:12.559872 29277 tablet_bootstrap.cc:654] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:12.560238 29278 tablet_bootstrap.cc:654] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:12.560664 29279 tablet_bootstrap.cc:654] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:12.560990 29278 log.cc:824] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:12.561022 29277 log.cc:824] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:12.561491 29279 log.cc:824] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:12.563115 29277 tablet_bootstrap.cc:492] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05: No bootstrap required, opened a new log
I20260809 04:49:12.563115 29278 tablet_bootstrap.cc:492] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01: No bootstrap required, opened a new log
I20260809 04:49:12.563146 29279 tablet_bootstrap.cc:492] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c: No bootstrap required, opened a new log
I20260809 04:49:12.563319 29278 ts_tablet_manager.cc:1397] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 04:49:12.563319 29277 ts_tablet_manager.cc:1397] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 04:49:12.563355 29279 ts_tablet_manager.cc:1397] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:49:12.567226 29279 raft_consensus.cc:348] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801db1f6ffcc4a4798de83c7bc79128c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 37793 } } peers { permanent_uuid: "9f6dbcd62224458d8fb27a92410f6e05" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 38425 } } peers { permanent_uuid: "633bfef3d9464c4994470a75e11e6c01" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34057 } }
I20260809 04:49:12.567512 29279 raft_consensus.cc:374] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:12.567576 29279 raft_consensus.cc:729] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801db1f6ffcc4a4798de83c7bc79128c, State: Initialized, Role: FOLLOWER
I20260809 04:49:12.567754 29278 raft_consensus.cc:348] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801db1f6ffcc4a4798de83c7bc79128c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 37793 } } peers { permanent_uuid: "9f6dbcd62224458d8fb27a92410f6e05" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 38425 } } peers { permanent_uuid: "633bfef3d9464c4994470a75e11e6c01" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34057 } }
I20260809 04:49:12.567754 29277 raft_consensus.cc:348] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801db1f6ffcc4a4798de83c7bc79128c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 37793 } } peers { permanent_uuid: "9f6dbcd62224458d8fb27a92410f6e05" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 38425 } } peers { permanent_uuid: "633bfef3d9464c4994470a75e11e6c01" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34057 } }
I20260809 04:49:12.568037 29277 raft_consensus.cc:374] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:12.568037 29278 raft_consensus.cc:374] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:12.568117 29277 raft_consensus.cc:729] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f6dbcd62224458d8fb27a92410f6e05, State: Initialized, Role: FOLLOWER
I20260809 04:49:12.568117 29278 raft_consensus.cc:729] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 633bfef3d9464c4994470a75e11e6c01, State: Initialized, Role: FOLLOWER
I20260809 04:49:12.568087 29279 consensus_queue.cc:260] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801db1f6ffcc4a4798de83c7bc79128c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 37793 } } peers { permanent_uuid: "9f6dbcd62224458d8fb27a92410f6e05" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 38425 } } peers { permanent_uuid: "633bfef3d9464c4994470a75e11e6c01" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34057 } }
I20260809 04:49:12.568570 29277 consensus_queue.cc:260] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801db1f6ffcc4a4798de83c7bc79128c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 37793 } } peers { permanent_uuid: "9f6dbcd62224458d8fb27a92410f6e05" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 38425 } } peers { permanent_uuid: "633bfef3d9464c4994470a75e11e6c01" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34057 } }
I20260809 04:49:12.568570 29278 consensus_queue.cc:260] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801db1f6ffcc4a4798de83c7bc79128c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 37793 } } peers { permanent_uuid: "9f6dbcd62224458d8fb27a92410f6e05" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 38425 } } peers { permanent_uuid: "633bfef3d9464c4994470a75e11e6c01" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34057 } }
I20260809 04:49:12.568976 29257 heartbeater.cc:503] Master 127.7.171.62:41527 was elected leader, sending a full tablet report...
I20260809 04:49:12.569239 29279 ts_tablet_manager.cc:1428] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:49:12.569545 29278 ts_tablet_manager.cc:1428] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260809 04:49:12.569553 29277 ts_tablet_manager.cc:1428] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260809 04:49:12.569873 28967 heartbeater.cc:503] Master 127.7.171.62:41527 was elected leader, sending a full tablet report...
I20260809 04:49:12.570245 29113 heartbeater.cc:503] Master 127.7.171.62:41527 was elected leader, sending a full tablet report...
W20260809 04:49:12.603010 28968 tablet.cc:2369] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:49:12.613458 29286 raft_consensus.cc:482] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:49:12.613731 29286 raft_consensus.cc:504] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801db1f6ffcc4a4798de83c7bc79128c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 37793 } } peers { permanent_uuid: "9f6dbcd62224458d8fb27a92410f6e05" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 38425 } } peers { permanent_uuid: "633bfef3d9464c4994470a75e11e6c01" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34057 } }
I20260809 04:49:12.615109 29286 leader_election.cc:288] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 801db1f6ffcc4a4798de83c7bc79128c (127.7.171.3:37793), 9f6dbcd62224458d8fb27a92410f6e05 (127.7.171.1:38425)
I20260809 04:49:12.618948 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.615463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:41643 (local address 127.7.171.3:37793)
0809 04:49:12.615596 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.615600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.615615 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:49:12.615781 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.615786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.615840 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.615932 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:12.616316 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.616841 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.617948 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.618340 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.618499 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.618562 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.618763 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.618834 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:49:12.618952 29115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.615361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:37793 (local address 127.7.171.2:41643)
0809 04:49:12.615514 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:12.615534 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:12.615647 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:12.615987 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:12.615990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:12.616010 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:12.616188 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.616194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.617035 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.617038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.617800 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.617817 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.618500 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.618503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.618611 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.618651 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 04:49:12.618742 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 04:49:12.618813 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 04:49:12.619659 29206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c748fa4482b14797a9fedf6aeea20658" candidate_uuid: "633bfef3d9464c4994470a75e11e6c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801db1f6ffcc4a4798de83c7bc79128c" is_pre_election: true
I20260809 04:49:12.619768 29276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.615852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:37359 (local address 127.7.171.1:38425)
0809 04:49:12.616027 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.616034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.616328 (+ 294us) server_negotiation.cc:407] Connection header received
0809 04:49:12.616514 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.616517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.616563 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.616650 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:12.616959 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.617499 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.618625 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.619128 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.619251 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.619318 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.619565 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.619641 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:49:12.619788 29287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.615716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:38425 (local address 127.7.171.2:37359)
0809 04:49:12.616232 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:12.616244 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:12.616366 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:12.616665 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:12.616669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:12.616687 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:12.616827 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.616832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.617617 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.617624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.618476 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:12.618485 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.619291 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.619295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:12.619407 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.619432 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:49:12.619568 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 04:49:12.619638 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":149,"threads_started":1}
I20260809 04:49:12.620024 29206 raft_consensus.cc:2454] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 633bfef3d9464c4994470a75e11e6c01 in term 0.
I20260809 04:49:12.620561 28989 leader_election.cc:302] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 633bfef3d9464c4994470a75e11e6c01, 801db1f6ffcc4a4798de83c7bc79128c; no voters:
I20260809 04:49:12.620617 28917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c748fa4482b14797a9fedf6aeea20658" candidate_uuid: "633bfef3d9464c4994470a75e11e6c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f6dbcd62224458d8fb27a92410f6e05" is_pre_election: true
I20260809 04:49:12.620831 29286 raft_consensus.cc:2781] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:49:12.620920 29286 raft_consensus.cc:482] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:49:12.620975 29286 raft_consensus.cc:3037] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:12.620951 28917 raft_consensus.cc:2454] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 633bfef3d9464c4994470a75e11e6c01 in term 0.
I20260809 04:49:12.622113 29286 raft_consensus.cc:504] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801db1f6ffcc4a4798de83c7bc79128c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 37793 } } peers { permanent_uuid: "9f6dbcd62224458d8fb27a92410f6e05" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 38425 } } peers { permanent_uuid: "633bfef3d9464c4994470a75e11e6c01" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34057 } }
I20260809 04:49:12.622478 29286 leader_election.cc:288] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [CANDIDATE]: Term 1 election: Requested vote from peers 801db1f6ffcc4a4798de83c7bc79128c (127.7.171.3:37793), 9f6dbcd62224458d8fb27a92410f6e05 (127.7.171.1:38425)
I20260809 04:49:12.622939 28917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c748fa4482b14797a9fedf6aeea20658" candidate_uuid: "633bfef3d9464c4994470a75e11e6c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f6dbcd62224458d8fb27a92410f6e05"
I20260809 04:49:12.622888 29206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c748fa4482b14797a9fedf6aeea20658" candidate_uuid: "633bfef3d9464c4994470a75e11e6c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801db1f6ffcc4a4798de83c7bc79128c"
I20260809 04:49:12.623080 29206 raft_consensus.cc:3037] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:12.623076 28917 raft_consensus.cc:3037] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:12.624221 29206 raft_consensus.cc:2454] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 633bfef3d9464c4994470a75e11e6c01 in term 1.
I20260809 04:49:12.624197 28917 raft_consensus.cc:2454] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 633bfef3d9464c4994470a75e11e6c01 in term 1.
I20260809 04:49:12.624573 28989 leader_election.cc:302] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 633bfef3d9464c4994470a75e11e6c01, 9f6dbcd62224458d8fb27a92410f6e05; no voters:
I20260809 04:49:12.624774 29286 raft_consensus.cc:2781] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:12.625044 29286 raft_consensus.cc:686] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 1 LEADER]: Becoming Leader. State: Replica: 633bfef3d9464c4994470a75e11e6c01, State: Running, Role: LEADER
I20260809 04:49:12.625430 29286 consensus_queue.cc:237] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801db1f6ffcc4a4798de83c7bc79128c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 37793 } } peers { permanent_uuid: "9f6dbcd62224458d8fb27a92410f6e05" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 38425 } } peers { permanent_uuid: "633bfef3d9464c4994470a75e11e6c01" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34057 } }
I20260809 04:49:12.628916 28760 catalog_manager.cc:5495] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 633bfef3d9464c4994470a75e11e6c01 (127.7.171.2). New cstate: current_term: 1 leader_uuid: "633bfef3d9464c4994470a75e11e6c01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801db1f6ffcc4a4798de83c7bc79128c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 37793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f6dbcd62224458d8fb27a92410f6e05" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 38425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "633bfef3d9464c4994470a75e11e6c01" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 34057 } health_report { overall_health: HEALTHY } } }
I20260809 04:49:12.655205 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.651277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41842 (local address 127.7.171.2:34057)
0809 04:49:12.651415 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.651420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.651435 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:49:12.651591 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.651594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.651652 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.651747 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:12.652011 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.652585 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.653560 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.653836 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.653958 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.654227 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:12.654681 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:12.654806 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.654938 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.655017 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:49:12.662065 29206 raft_consensus.cc:1261] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c [term 1 FOLLOWER]: Refusing update from remote peer 633bfef3d9464c4994470a75e11e6c01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:12.662168 28917 raft_consensus.cc:1261] T c748fa4482b14797a9fedf6aeea20658 P 9f6dbcd62224458d8fb27a92410f6e05 [term 1 FOLLOWER]: Refusing update from remote peer 633bfef3d9464c4994470a75e11e6c01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:12.663002 29288 consensus_queue.cc:1035] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801db1f6ffcc4a4798de83c7bc79128c" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 37793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:12.663200 29286 consensus_queue.cc:1035] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f6dbcd62224458d8fb27a92410f6e05" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 38425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:12.672305 29293 mvcc.cc:205] Tried to move back new op lower bound from 7316483902095646720 to 7316483901956657152. Current Snapshot: MvccSnapshot[applied={T|T < 7316483902095646720}]
I20260809 04:49:12.681854 28763 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:12.682133 28763 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:12.684957 28763 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b12d60b882f43ed8c0c785d6847ba7a: Sending DeleteTablet for 3 replicas of tablet c748fa4482b14797a9fedf6aeea20658
I20260809 04:49:12.685971 29186 tablet_service.cc:1514] Processing DeleteTablet for tablet c748fa4482b14797a9fedf6aeea20658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:12 UTC) from {username='slave'} at 127.0.0.1:52456
I20260809 04:49:12.685992 28897 tablet_service.cc:1514] Processing DeleteTablet for tablet c748fa4482b14797a9fedf6aeea20658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:12 UTC) from {username='slave'} at 127.0.0.1:53870
I20260809 04:49:12.686120 29038 tablet_service.cc:1514] Processing DeleteTablet for tablet c748fa4482b14797a9fedf6aeea20658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:12 UTC) from {username='slave'} at 127.0.0.1:41838
I20260809 04:49:12.686642 29303 tablet_replica.cc:330] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c: stopping tablet replica
I20260809 04:49:12.686672 29304 tablet_replica.cc:330] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01: stopping tablet replica
I20260809 04:49:12.686980 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 28828
I20260809 04:49:12.687003 29304 raft_consensus.cc:2229] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:49:12.687003 29303 raft_consensus.cc:2229] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:49:12.688136 29303 raft_consensus.cc:2258] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:12.688378 29304 raft_consensus.cc:2258] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:12.689937 29304 ts_tablet_manager.cc:1905] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:12.690096 29303 ts_tablet_manager.cc:1905] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:12.692965 29303 ts_tablet_manager.cc:1918] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:12.692965 29304 ts_tablet_manager.cc:1918] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:12.693074 29304 log.cc:1196] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/c748fa4482b14797a9fedf6aeea20658
I20260809 04:49:12.693219 29303 log.cc:1196] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/c748fa4482b14797a9fedf6aeea20658
I20260809 04:49:12.693413 29304 ts_tablet_manager.cc:1939] T c748fa4482b14797a9fedf6aeea20658 P 633bfef3d9464c4994470a75e11e6c01: Deleting consensus metadata
I20260809 04:49:12.693687 29303 ts_tablet_manager.cc:1939] T c748fa4482b14797a9fedf6aeea20658 P 801db1f6ffcc4a4798de83c7bc79128c: Deleting consensus metadata
I20260809 04:49:12.694777 28747 catalog_manager.cc:4841] TS 633bfef3d9464c4994470a75e11e6c01 (127.7.171.2:34057): tablet c748fa4482b14797a9fedf6aeea20658 (table test-table [id=bd5b894108c24011af1cc266c95180fc]) successfully deleted
I20260809 04:49:12.694826 28748 catalog_manager.cc:4841] TS 801db1f6ffcc4a4798de83c7bc79128c (127.7.171.3:37793): tablet c748fa4482b14797a9fedf6aeea20658 (table test-table [id=bd5b894108c24011af1cc266c95180fc]) successfully deleted
W20260809 04:49:12.700381 28748 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:38425 (error 108)
I20260809 04:49:12.700470 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 28970
I20260809 04:49:12.701803 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.701396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:38425 (local address 127.0.0.1:53882)
0809 04:49:12.701622 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:12.701707 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:38425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260809 04:49:12.702150 28748 catalog_manager.cc:4584] TS 9f6dbcd62224458d8fb27a92410f6e05 (127.7.171.1:38425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c748fa4482b14797a9fedf6aeea20658: Network error: Client connection negotiation failed: client connection to 127.7.171.1:38425: connect: Connection refused (error 111)
I20260809 04:49:12.708506 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 29117
I20260809 04:49:12.716588 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 28730
2026-08-09T04:49:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1152 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:12 dist-test-slave-1zqn krb5kdc[29309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:12 dist-test-slave-1zqn krb5kdc[29309](info): set up 2 sockets
Aug 09 04:49:12 dist-test-slave-1zqn krb5kdc[29309](info): commencing operation
krb5kdc: starting...
W20260809 04:49:13.827813 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:13 dist-test-slave-1zqn krb5kdc[29309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:49:13.967756 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46279
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:46279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:14.082603 29323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:14.082947 29323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:14.083062 29323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:14.087333 29323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:14.087440 29323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:14.087477 29323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:14.087520 29323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:14.087558 29323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:14.093056 29323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:46279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:14.094280 29323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:14.095369 29323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:14.104350 29329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:14.104346 29328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:14.104351 29331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:14.104619 29323 server_base.cc:1034] running on GCE node
I20260809 04:49:14.105376 29323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:14.106563 29323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:14.107767 29323 hybrid_clock.cc:648] HybridClock initialized: now 1786250954107738 us; error 43 us; skew 500 ppm
Aug 09 04:49:14 dist-test-slave-1zqn krb5kdc[29309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250954, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:14.110718 29323 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:49:14.111567 29323 webserver.cc:460] Webserver started at http://127.7.171.62:44099/ using document root <none> and password file <none>
I20260809 04:49:14.112150 29323 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:14.112219 29323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:14.112388 29323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:14.114413 29323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "56deebd60bf7448ab2a22d95d064d7dc"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "95a4fd54da0f67fe55af7e269f398d3d"
server_key_iv: "c090c75e3cac4b0ac7ea35051f924e1e"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.115146 29323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "56deebd60bf7448ab2a22d95d064d7dc"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "95a4fd54da0f67fe55af7e269f398d3d"
server_key_iv: "c090c75e3cac4b0ac7ea35051f924e1e"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.119107 29323 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:49:14.121730 29338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:14.123070 29323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:49:14.123208 29323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "56deebd60bf7448ab2a22d95d064d7dc"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "95a4fd54da0f67fe55af7e269f398d3d"
server_key_iv: "c090c75e3cac4b0ac7ea35051f924e1e"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.123325 29323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:14.143667 29323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:14.147060 29323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:14.147272 29323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:14.155369 29323 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:46279
I20260809 04:49:14.155396 29394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:46279 every 8 connection(s)
I20260809 04:49:14.156459 29323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:14.159384 29395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:14.165270 29395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc: Bootstrap starting.
I20260809 04:49:14.165457 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 29323
I20260809 04:49:14.165618 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:14.165911 7852 external_mini_cluster.cc:1428] Setting key bf8ed77ef0254dd47f85540cb513a717
I20260809 04:49:14.168195 29395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:14.169107 29395 log.cc:824] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:14.171218 29395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc: No bootstrap required, opened a new log
I20260809 04:49:14.174957 29395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56deebd60bf7448ab2a22d95d064d7dc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46279 } }
I20260809 04:49:14.175268 29395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:14.175338 29395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56deebd60bf7448ab2a22d95d064d7dc, State: Initialized, Role: FOLLOWER
Aug 09 04:49:14 dist-test-slave-1zqn krb5kdc[29309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:14.175853 29395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56deebd60bf7448ab2a22d95d064d7dc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46279 } }
I20260809 04:49:14.176033 29395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:14.176143 29395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:14.176267 29395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:14.177459 29395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56deebd60bf7448ab2a22d95d064d7dc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46279 } }
I20260809 04:49:14.177820 29395 leader_election.cc:302] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56deebd60bf7448ab2a22d95d064d7dc; no voters:
I20260809 04:49:14.178166 29395 leader_election.cc:288] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:14.178304 29400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:14.178561 29400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [term 1 LEADER]: Becoming Leader. State: Replica: 56deebd60bf7448ab2a22d95d064d7dc, State: Running, Role: LEADER
I20260809 04:49:14.178920 29400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56deebd60bf7448ab2a22d95d064d7dc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46279 } }
I20260809 04:49:14.179656 29395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:14.180770 29401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56deebd60bf7448ab2a22d95d064d7dc. Latest consensus state: current_term: 1 leader_uuid: "56deebd60bf7448ab2a22d95d064d7dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56deebd60bf7448ab2a22d95d064d7dc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46279 } } }
I20260809 04:49:14.180900 29401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:14.181358 29409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:14.181784 29402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56deebd60bf7448ab2a22d95d064d7dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56deebd60bf7448ab2a22d95d064d7dc" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46279 } } }
I20260809 04:49:14.181896 29402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:14.182515 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.167473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.7.171.62:46279)
0809 04:49:14.167909 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:14.167930 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:14.167975 (+ 45us) server_negotiation.cc:407] Connection header received
0809 04:49:14.168914 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:14.168942 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:14.169379 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:14.169801 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:14.170638 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.171705 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.172670 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.173165 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.173347 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.176252 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:14.176258 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:14.176283 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:14.176312 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:14.178733 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:14.179271 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:14.179277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:14.179283 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:14.179437 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:14.179763 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:14.179766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:14.179768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:14.180348 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:14.180558 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:14.181210 (+ 652us) server_negotiation.cc:299] Negotiation successful
0809 04:49:14.181524 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":119,"threads_started":1}
I20260809 04:49:14.184791 29409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:14.190841 29409 catalog_manager.cc:1347] Generated new cluster ID: d553d4fb2c3145cdad520f6c2c9df2d2
I20260809 04:49:14.190922 29409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:14.207799 29409 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:14.209251 29409 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:14.217304 29409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc: Generated new TSK 0
I20260809 04:49:14.218143 29409 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.171.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
Entry for principal kudu/127.7.171.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab.
I20260809 04:49:14.261703 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:46279
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:14.377744 29422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:14.378044 29422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:14.378129 29422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:14.382453 29422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:14.382540 29422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:14.382624 29422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:49:14.389163 29422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.1.keytab
--principal=kudu/127.7.171.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:14.390244 29422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:14.391265 29422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:14.401054 29430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:14.401136 29427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:14.401048 29428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:14.401340 29422 server_base.cc:1034] running on GCE node
I20260809 04:49:14.401770 29422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:14.402463 29422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:14.403669 29422 hybrid_clock.cc:648] HybridClock initialized: now 1786250954403632 us; error 55 us; skew 500 ppm
Aug 09 04:49:14 dist-test-slave-1zqn krb5kdc[29309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250954, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:14.406728 29422 init.cc:377] Logged in from keytab as kudu/127.7.171.1@KRBTEST.COM (short username kudu)
I20260809 04:49:14.407640 29422 webserver.cc:460] Webserver started at http://127.7.171.1:42707/ using document root <none> and password file <none>
I20260809 04:49:14.408420 29422 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:14.408524 29422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:14.408785 29422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:14.410871 29422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "fa85a3196a3e48dab1d8d44fde9c2267"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "a97a110aa00ce6dd943f6adbf1cb5fe9"
server_key_iv: "ff367dd6803d7172c487ecfca0031b71"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.411484 29422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "fa85a3196a3e48dab1d8d44fde9c2267"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "a97a110aa00ce6dd943f6adbf1cb5fe9"
server_key_iv: "ff367dd6803d7172c487ecfca0031b71"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.415377 29422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260809 04:49:14.418051 29438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:14.419402 29422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:49:14.419577 29422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "fa85a3196a3e48dab1d8d44fde9c2267"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "a97a110aa00ce6dd943f6adbf1cb5fe9"
server_key_iv: "ff367dd6803d7172c487ecfca0031b71"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.419705 29422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:14.446220 29422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:14.449721 29422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:14.450047 29422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:14.450759 29422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:14.451707 29422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:14.451790 29422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:14.451874 29422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:14.451923 29422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:14.463903 29422 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:36333
I20260809 04:49:14.463994 29565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:36333 every 8 connection(s)
I20260809 04:49:14.465341 29422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:49:14.469941 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 29422
I20260809 04:49:14.470088 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:49:14.470379 7852 external_mini_cluster.cc:1428] Setting key 83503b208a26ccf7be1540f1dbe175c3
Aug 09 04:49:14 dist-test-slave-1zqn krb5kdc[29309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250954, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.1@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.171.2@KRBTEST.COM; defaulting to no policy
I20260809 04:49:14.481878 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.468121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:40355 (local address 127.7.171.62:46279)
0809 04:49:14.468275 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:14.468279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:14.469361 (+ 1082us) server_negotiation.cc:407] Connection header received
0809 04:49:14.470779 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:14.470783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:14.470843 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:14.470923 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:14.472468 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.473266 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.474659 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.474899 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.475001 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.477937 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:14.477941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:14.477943 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:14.477966 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:14.479563 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:14.480131 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:14.480139 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:14.480141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:14.480216 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:14.480620 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:14.480624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:14.480625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:14.480883 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:14.480991 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:14.481565 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 04:49:14.481699 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:49:14.482836 29569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.468625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:46279 (local address 127.7.171.1:40355)
0809 04:49:14.469204 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:14.469250 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:14.470551 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:14.471143 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:14.471151 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:14.471661 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:14.472313 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.472328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.473406 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.473409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.474523 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.474537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.475073 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.475079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.475313 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.476120 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:14.476146 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:14.477728 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:14.479687 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:14.479693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:14.479708 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:14.479992 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:14.480312 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:14.480316 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:14.480318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:14.480517 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:14.480991 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:14.480998 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:14.481313 (+ 315us) client_negotiation.cc:769] Sending connection context
0809 04:49:14.481507 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 04:49:14.481845 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.2@KRBTEST.COM" created.
I20260809 04:49:14.484279 29566 heartbeater.cc:347] Connected to a master server at 127.7.171.62:46279
I20260809 04:49:14.484577 29566 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:14.485142 29566 heartbeater.cc:511] Master 127.7.171.62:46279 requested a full tablet report, sending...
I20260809 04:49:14.486801 29356 ts_manager.cc:194] Registered new tserver with Master: fa85a3196a3e48dab1d8d44fde9c2267 (127.7.171.1:36333)
I20260809 04:49:14.488456 29356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.1@KRBTEST.COM'} at 127.7.171.1:40355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
Entry for principal kudu/127.7.171.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab.
I20260809 04:49:14.503808 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:46279
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:14.616860 29572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:14.617126 29572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:14.617169 29572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:14.621115 29572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:14.621220 29572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:14.621313 29572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:49:14.626690 29572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.2.keytab
--principal=kudu/127.7.171.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:14.627818 29572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:14.629129 29572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:14.639158 29577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:14.639114 29578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:14.639078 29580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:14.639371 29572 server_base.cc:1034] running on GCE node
I20260809 04:49:14.639727 29572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:14.640448 29572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:14.641638 29572 hybrid_clock.cc:648] HybridClock initialized: now 1786250954641620 us; error 37 us; skew 500 ppm
Aug 09 04:49:14 dist-test-slave-1zqn krb5kdc[29309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250954, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:14.644575 29572 init.cc:377] Logged in from keytab as kudu/127.7.171.2@KRBTEST.COM (short username kudu)
I20260809 04:49:14.645455 29572 webserver.cc:460] Webserver started at http://127.7.171.2:36549/ using document root <none> and password file <none>
I20260809 04:49:14.645982 29572 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:14.646044 29572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:14.646209 29572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:14.648283 29572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "28044201fe7c47928c6337921990acbc"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "96f72bc89901d5727eda9a0e300705ce"
server_key_iv: "8863e865cf5114558d283c1685f28c18"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.648813 29572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "28044201fe7c47928c6337921990acbc"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "96f72bc89901d5727eda9a0e300705ce"
server_key_iv: "8863e865cf5114558d283c1685f28c18"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.652730 29572 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:49:14.655366 29587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:14.656850 29572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:14.656951 29572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "28044201fe7c47928c6337921990acbc"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "96f72bc89901d5727eda9a0e300705ce"
server_key_iv: "8863e865cf5114558d283c1685f28c18"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.657037 29572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:14.676008 29572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:14.679616 29572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:14.679783 29572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:14.680576 29572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:14.681593 29572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:14.681644 29572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:14.681685 29572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:14.681701 29572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:14.691203 29572 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:42769
I20260809 04:49:14.691265 29709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:42769 every 8 connection(s)
I20260809 04:49:14.692194 29572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:49:14.701481 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 29572
I20260809 04:49:14.701635 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:49:14.701953 7852 external_mini_cluster.cc:1428] Setting key bcdd01e2b32bff5854f0b0241a2d2fe4
Aug 09 04:49:14 dist-test-slave-1zqn krb5kdc[29309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250954, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.2@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:14.707129 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.694133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:37735 (local address 127.7.171.62:46279)
0809 04:49:14.694298 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:14.694303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:14.695445 (+ 1142us) server_negotiation.cc:407] Connection header received
0809 04:49:14.696465 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:14.696467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:14.696535 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:14.696625 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:14.697948 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.698480 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.699738 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.699944 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.700025 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.702862 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:14.702865 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:14.702868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:14.702892 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:14.704655 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:14.705331 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:14.705333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:14.705334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:14.705407 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:14.705811 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:14.705814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:14.705815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:14.706114 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:14.706224 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:14.706795 (+ 571us) server_negotiation.cc:299] Negotiation successful
0809 04:49:14.706932 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:49:14.707903 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.694587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:46279 (local address 127.7.171.2:37735)
0809 04:49:14.695287 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:14.695332 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:14.696253 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:14.696849 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:14.696857 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:14.697309 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:14.697809 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.697823 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.698644 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.698648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.699616 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.699627 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.700158 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.700163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.700313 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.700898 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:14.700917 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:14.702685 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:14.704777 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:14.704786 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:14.704805 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:14.705216 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:14.705503 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:14.705509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:14.705513 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:14.705711 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:14.706255 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:14.706261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:14.706510 (+ 249us) client_negotiation.cc:769] Sending connection context
0809 04:49:14.706719 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 04:49:14.706992 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":107,"threads_started":1}
I20260809 04:49:14.709225 29711 heartbeater.cc:347] Connected to a master server at 127.7.171.62:46279
I20260809 04:49:14.709509 29711 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:14.710143 29711 heartbeater.cc:511] Master 127.7.171.62:46279 requested a full tablet report, sending...
I20260809 04:49:14.711355 29356 ts_manager.cc:194] Registered new tserver with Master: 28044201fe7c47928c6337921990acbc (127.7.171.2:42769)
I20260809 04:49:14.711937 29356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.2@KRBTEST.COM'} at 127.7.171.2:37735
WARNING: no policy specified for kudu/127.7.171.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
Entry for principal kudu/127.7.171.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab.
I20260809 04:49:14.735523 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:46279
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:14.849637 29716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:14.849946 29716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:14.850029 29716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:14.854116 29716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:14.854208 29716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:14.854302 29716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:49:14.859633 29716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.3.keytab
--principal=kudu/127.7.171.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:14.860741 29716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:14.861940 29716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:14.871671 29721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:14.871682 29724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:14.871748 29716 server_base.cc:1034] running on GCE node
W20260809 04:49:14.871699 29722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:14.872435 29716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:14.873020 29716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:14.874195 29716 hybrid_clock.cc:648] HybridClock initialized: now 1786250954874184 us; error 30 us; skew 500 ppm
Aug 09 04:49:14 dist-test-slave-1zqn krb5kdc[29309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250954, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:14.877122 29716 init.cc:377] Logged in from keytab as kudu/127.7.171.3@KRBTEST.COM (short username kudu)
I20260809 04:49:14.878023 29716 webserver.cc:460] Webserver started at http://127.7.171.3:40919/ using document root <none> and password file <none>
I20260809 04:49:14.878603 29716 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:14.878710 29716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:14.878952 29716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:14.881287 29716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "55bcc2348bcf458db7a30d04ee6db051"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "b0e1f4bbfcc1235cebd8be54dce121db"
server_key_iv: "0b24cb058369c5ca10c8c22769e566d7"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.881894 29716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "55bcc2348bcf458db7a30d04ee6db051"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "b0e1f4bbfcc1235cebd8be54dce121db"
server_key_iv: "0b24cb058369c5ca10c8c22769e566d7"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.886039 29716 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:49:14.888567 29733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:14.889902 29716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:14.890002 29716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "55bcc2348bcf458db7a30d04ee6db051"
format_stamp: "Formatted at 2026-08-09 04:49:14 on dist-test-slave-1zqn"
server_key: "b0e1f4bbfcc1235cebd8be54dce121db"
server_key_iv: "0b24cb058369c5ca10c8c22769e566d7"
server_key_version: "encryptionkey@0"
I20260809 04:49:14.890178 29716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:14.911381 29716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:14.914546 29716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:14.914755 29716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:14.915374 29716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:14.916271 29716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:14.916321 29716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:14.916384 29716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:14.916419 29716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:14.926301 29716 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:34867
I20260809 04:49:14.926393 29859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:34867 every 8 connection(s)
I20260809 04:49:14.927312 29716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:49:14.933086 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 29716
I20260809 04:49:14.933246 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:49:14.933526 7852 external_mini_cluster.cc:1428] Setting key 9acbde91d6eb0976c1f2947ef6cb0bf1
Aug 09 04:49:14 dist-test-slave-1zqn krb5kdc[29309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250954, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.3@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:14.941676 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.929551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:42351 (local address 127.7.171.62:46279)
0809 04:49:14.929713 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:14.929718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:14.930697 (+ 979us) server_negotiation.cc:407] Connection header received
0809 04:49:14.931545 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:14.931548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:14.931595 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:14.931669 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:14.932867 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.933506 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.934912 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.935227 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.935336 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.938118 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:14.938121 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:14.938123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:14.938146 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:14.939572 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:14.940259 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:14.940262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:14.940263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:14.940337 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:14.940681 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:14.940683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:14.940684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:14.940849 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:14.940921 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:14.941460 (+ 539us) server_negotiation.cc:299] Negotiation successful
0809 04:49:14.941558 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:49:14.942494 29862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.930055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:46279 (local address 127.7.171.3:42351)
0809 04:49:14.930534 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:14.930578 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:14.931340 (+ 762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:14.931827 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:14.931833 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:14.932227 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:14.932735 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.932749 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.933665 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.933669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.934789 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.934804 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.935396 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.935399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.935550 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.936239 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:14.936255 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:14.937876 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:14.939689 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:14.939694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:14.939706 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:14.940164 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:14.940425 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:14.940428 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:14.940429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:14.940586 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:14.940999 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:14.941004 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:14.941236 (+ 232us) client_negotiation.cc:769] Sending connection context
0809 04:49:14.941394 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 04:49:14.941659 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":117,"threads_started":1}
I20260809 04:49:14.943610 29860 heartbeater.cc:347] Connected to a master server at 127.7.171.62:46279
I20260809 04:49:14.943852 29860 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:14.944494 29860 heartbeater.cc:511] Master 127.7.171.62:46279 requested a full tablet report, sending...
I20260809 04:49:14.945542 29356 ts_manager.cc:194] Registered new tserver with Master: 55bcc2348bcf458db7a30d04ee6db051 (127.7.171.3:34867)
I20260809 04:49:14.946170 29356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.171.3@KRBTEST.COM'} at 127.7.171.3:42351
I20260809 04:49:14.947522 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:49:14.957437 29398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.949518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58256 (local address 127.7.171.62:46279)
0809 04:49:14.949663 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:14.949667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:14.949873 (+ 206us) server_negotiation.cc:407] Connection header received
0809 04:49:14.950021 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:14.950023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:14.950088 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:14.950154 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:14.950813 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.951321 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.952233 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.952433 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.952511 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.954089 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:14.954092 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:14.954094 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:14.954116 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:14.955656 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:14.956281 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:14.956284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:14.956285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:14.956363 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:14.956654 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:14.956656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:14.956657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:14.956861 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:14.956962 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:14.957151 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 04:49:14.957243 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:49:14.960297 29356 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:49:14.963322 29356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:49:14.977941 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.971710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:42769 (local address 127.0.0.1:50114)
0809 04:49:14.972102 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:14.972155 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:14.972332 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:14.973139 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:14.973146 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:14.973173 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:14.973304 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.973311 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.974791 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.974794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.976436 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.976456 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.977393 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.977396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.977503 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.977539 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 04:49:14.977646 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 04:49:14.977773 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":214,"thread_start_us":75,"threads_started":1}
I20260809 04:49:14.977950 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.971853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.7.171.2:42769)
0809 04:49:14.972279 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:14.972291 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:14.972317 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:49:14.972592 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:14.972600 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:14.972828 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:14.972998 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:14.973475 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.974649 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.976582 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.977263 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.977394 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.977550 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:14.977688 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 04:49:14.977783 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":152,"threads_started":1}
I20260809 04:49:14.978408 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.972843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36333 (local address 127.0.0.1:34284)
0809 04:49:14.973594 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:14.973607 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:14.973704 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:14.974261 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:14.974263 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:14.974281 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:14.974367 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.974373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.975456 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.975461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.976733 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.976743 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.977915 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.977918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.978055 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.978088 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 04:49:14.978209 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 04:49:14.978275 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"mutex_wait_us":1,"thread_start_us":113,"threads_started":1}
I20260809 04:49:14.978645 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.972480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:34867 (local address 127.0.0.1:39170)
0809 04:49:14.973423 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:14.973438 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:14.973527 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:14.974069 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:14.974072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:14.974090 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:14.974175 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.974181 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.975505 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.975509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.976614 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:14.976622 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.978227 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.978231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:14.978326 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.978358 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:49:14.978439 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 04:49:14.978506 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"thread_start_us":110,"threads_started":1}
I20260809 04:49:14.978889 29878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.973027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34284 (local address 127.7.171.1:36333)
0809 04:49:14.973340 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:14.973361 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:14.973704 (+ 343us) server_negotiation.cc:407] Connection header received
0809 04:49:14.973816 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:14.973822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:14.974018 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:14.974161 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:14.974485 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.975323 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.976864 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.977666 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.977759 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.977880 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:14.978599 (+ 719us) server_negotiation.cc:299] Negotiation successful
0809 04:49:14.978734 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":88,"threads_started":1}
I20260809 04:49:14.979195 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.972679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39170 (local address 127.7.171.3:34867)
0809 04:49:14.973661 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:14.973674 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:14.973697 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:49:14.973777 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:14.973786 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:14.973938 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:14.974097 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:14.974309 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.975334 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.976724 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:14.977568 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:14.977681 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:14.977835 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:14.978650 (+ 815us) server_negotiation.cc:299] Negotiation successful
0809 04:49:14.978875 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":871,"thread_start_us":150,"threads_started":1}
I20260809 04:49:14.981375 29637 tablet_service.cc:1467] Processing CreateTablet for tablet 2dc5740a1d7c4af7a455f8c230e71437 (DEFAULT_TABLE table=test-table [id=825b18569e67486d8e7599aa7add2e59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:14.981808 29783 tablet_service.cc:1467] Processing CreateTablet for tablet 2dc5740a1d7c4af7a455f8c230e71437 (DEFAULT_TABLE table=test-table [id=825b18569e67486d8e7599aa7add2e59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:14.982033 29490 tablet_service.cc:1467] Processing CreateTablet for tablet 2dc5740a1d7c4af7a455f8c230e71437 (DEFAULT_TABLE table=test-table [id=825b18569e67486d8e7599aa7add2e59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:14.982563 29637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dc5740a1d7c4af7a455f8c230e71437. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:14.982812 29783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dc5740a1d7c4af7a455f8c230e71437. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:14.983021 29490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dc5740a1d7c4af7a455f8c230e71437. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:14.988888 29879 tablet_bootstrap.cc:492] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: Bootstrap starting.
I20260809 04:49:14.990576 29882 tablet_bootstrap.cc:492] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267: Bootstrap starting.
I20260809 04:49:14.991417 29879 tablet_bootstrap.cc:654] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:14.992414 29881 tablet_bootstrap.cc:492] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: Bootstrap starting.
I20260809 04:49:14.992609 29879 log.cc:824] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:14.992647 29882 tablet_bootstrap.cc:654] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:14.993489 29882 log.cc:824] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:14.994644 29879 tablet_bootstrap.cc:492] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: No bootstrap required, opened a new log
I20260809 04:49:14.994833 29879 ts_tablet_manager.cc:1397] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:49:14.995041 29881 tablet_bootstrap.cc:654] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:14.995564 29882 tablet_bootstrap.cc:492] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267: No bootstrap required, opened a new log
I20260809 04:49:14.995729 29882 ts_tablet_manager.cc:1397] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:49:14.995853 29881 log.cc:824] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:14.997462 29881 tablet_bootstrap.cc:492] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: No bootstrap required, opened a new log
I20260809 04:49:14.997598 29881 ts_tablet_manager.cc:1397] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 04:49:14.998895 29879 raft_consensus.cc:348] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28044201fe7c47928c6337921990acbc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42769 } } peers { permanent_uuid: "55bcc2348bcf458db7a30d04ee6db051" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 34867 } } peers { permanent_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36333 } }
I20260809 04:49:14.999166 29879 raft_consensus.cc:374] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:14.999225 29879 raft_consensus.cc:729] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55bcc2348bcf458db7a30d04ee6db051, State: Initialized, Role: FOLLOWER
I20260809 04:49:14.999681 29879 consensus_queue.cc:260] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28044201fe7c47928c6337921990acbc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42769 } } peers { permanent_uuid: "55bcc2348bcf458db7a30d04ee6db051" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 34867 } } peers { permanent_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36333 } }
I20260809 04:49:15.000099 29882 raft_consensus.cc:348] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28044201fe7c47928c6337921990acbc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42769 } } peers { permanent_uuid: "55bcc2348bcf458db7a30d04ee6db051" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 34867 } } peers { permanent_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36333 } }
I20260809 04:49:15.000334 29882 raft_consensus.cc:374] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:15.000379 29882 raft_consensus.cc:729] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa85a3196a3e48dab1d8d44fde9c2267, State: Initialized, Role: FOLLOWER
I20260809 04:49:15.000402 29860 heartbeater.cc:503] Master 127.7.171.62:46279 was elected leader, sending a full tablet report...
I20260809 04:49:15.000797 29879 ts_tablet_manager.cc:1428] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 04:49:15.000792 29882 consensus_queue.cc:260] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28044201fe7c47928c6337921990acbc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42769 } } peers { permanent_uuid: "55bcc2348bcf458db7a30d04ee6db051" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 34867 } } peers { permanent_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36333 } }
I20260809 04:49:15.000794 29881 raft_consensus.cc:348] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28044201fe7c47928c6337921990acbc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42769 } } peers { permanent_uuid: "55bcc2348bcf458db7a30d04ee6db051" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 34867 } } peers { permanent_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36333 } }
I20260809 04:49:15.001052 29881 raft_consensus.cc:374] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:15.001112 29881 raft_consensus.cc:729] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28044201fe7c47928c6337921990acbc, State: Initialized, Role: FOLLOWER
I20260809 04:49:15.001675 29566 heartbeater.cc:503] Master 127.7.171.62:46279 was elected leader, sending a full tablet report...
I20260809 04:49:15.001772 29882 ts_tablet_manager.cc:1428] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:49:15.001513 29881 consensus_queue.cc:260] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28044201fe7c47928c6337921990acbc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42769 } } peers { permanent_uuid: "55bcc2348bcf458db7a30d04ee6db051" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 34867 } } peers { permanent_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36333 } }
I20260809 04:49:15.003182 29711 heartbeater.cc:503] Master 127.7.171.62:46279 was elected leader, sending a full tablet report...
I20260809 04:49:15.003400 29881 ts_tablet_manager.cc:1428] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260809 04:49:15.158397 29886 raft_consensus.cc:482] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:49:15.158653 29886 raft_consensus.cc:504] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28044201fe7c47928c6337921990acbc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42769 } } peers { permanent_uuid: "55bcc2348bcf458db7a30d04ee6db051" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 34867 } } peers { permanent_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36333 } }
I20260809 04:49:15.160073 29886 leader_election.cc:288] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28044201fe7c47928c6337921990acbc (127.7.171.2:42769), fa85a3196a3e48dab1d8d44fde9c2267 (127.7.171.1:36333)
I20260809 04:49:15.164834 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.160549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:46297 (local address 127.7.171.2:42769)
0809 04:49:15.160715 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:15.160721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:15.160739 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:49:15.160805 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:15.160811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:15.160892 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:15.160998 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:15.161470 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.162203 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.163580 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.164246 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.164353 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.164418 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:15.164624 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 04:49:15.164705 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:49:15.164825 29862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.160388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:42769 (local address 127.7.171.3:46297)
0809 04:49:15.160531 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:15.160550 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:15.160658 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:15.161173 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:15.161178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:15.161203 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:15.161343 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:15.161358 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.162323 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.162328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:15.163429 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:15.163439 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.164375 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.164378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:15.164488 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.164515 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:49:15.164597 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 04:49:15.164677 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"spinlock_wait_cycles":2432}
I20260809 04:49:15.165524 29878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.161018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:38095 (local address 127.7.171.1:36333)
0809 04:49:15.161133 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:15.161137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:15.161205 (+ 68us) server_negotiation.cc:407] Connection header received
0809 04:49:15.161326 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:15.161329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:15.161372 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:15.161441 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:15.161856 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.162618 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.163936 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.164510 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.164622 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.164685 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:15.165311 (+ 626us) server_negotiation.cc:299] Negotiation successful
0809 04:49:15.165407 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:49:15.165617 29659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc5740a1d7c4af7a455f8c230e71437" candidate_uuid: "55bcc2348bcf458db7a30d04ee6db051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28044201fe7c47928c6337921990acbc" is_pre_election: true
I20260809 04:49:15.165546 29889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.160706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36333 (local address 127.7.171.3:38095)
0809 04:49:15.161132 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:15.161146 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:15.161241 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:15.161536 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:15.161539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:15.161557 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:15.161677 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:15.161684 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.162748 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.162751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:15.163812 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:15.163820 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.164757 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.164761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:15.164859 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.165191 (+ 332us) client_negotiation.cc:769] Sending connection context
0809 04:49:15.165296 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 04:49:15.165408 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":128,"threads_started":1}
I20260809 04:49:15.165988 29659 raft_consensus.cc:2454] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55bcc2348bcf458db7a30d04ee6db051 in term 0.
I20260809 04:49:15.166396 29506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc5740a1d7c4af7a455f8c230e71437" candidate_uuid: "55bcc2348bcf458db7a30d04ee6db051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" is_pre_election: true
I20260809 04:49:15.166482 29737 leader_election.cc:302] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28044201fe7c47928c6337921990acbc, 55bcc2348bcf458db7a30d04ee6db051; no voters:
I20260809 04:49:15.166726 29506 raft_consensus.cc:2454] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55bcc2348bcf458db7a30d04ee6db051 in term 0.
I20260809 04:49:15.166772 29886 raft_consensus.cc:2781] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:49:15.166859 29886 raft_consensus.cc:482] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:49:15.166914 29886 raft_consensus.cc:3037] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:15.168308 29886 raft_consensus.cc:504] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28044201fe7c47928c6337921990acbc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42769 } } peers { permanent_uuid: "55bcc2348bcf458db7a30d04ee6db051" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 34867 } } peers { permanent_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36333 } }
I20260809 04:49:15.168653 29886 leader_election.cc:288] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [CANDIDATE]: Term 1 election: Requested vote from peers 28044201fe7c47928c6337921990acbc (127.7.171.2:42769), fa85a3196a3e48dab1d8d44fde9c2267 (127.7.171.1:36333)
I20260809 04:49:15.169051 29659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc5740a1d7c4af7a455f8c230e71437" candidate_uuid: "55bcc2348bcf458db7a30d04ee6db051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28044201fe7c47928c6337921990acbc"
I20260809 04:49:15.169199 29659 raft_consensus.cc:3037] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:15.169267 29506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dc5740a1d7c4af7a455f8c230e71437" candidate_uuid: "55bcc2348bcf458db7a30d04ee6db051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa85a3196a3e48dab1d8d44fde9c2267"
I20260809 04:49:15.169399 29506 raft_consensus.cc:3037] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:15.170377 29659 raft_consensus.cc:2454] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55bcc2348bcf458db7a30d04ee6db051 in term 1.
I20260809 04:49:15.170657 29506 raft_consensus.cc:2454] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55bcc2348bcf458db7a30d04ee6db051 in term 1.
I20260809 04:49:15.170966 29737 leader_election.cc:302] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55bcc2348bcf458db7a30d04ee6db051, fa85a3196a3e48dab1d8d44fde9c2267; no voters:
I20260809 04:49:15.171176 29886 raft_consensus.cc:2781] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:15.171418 29886 raft_consensus.cc:686] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 1 LEADER]: Becoming Leader. State: Replica: 55bcc2348bcf458db7a30d04ee6db051, State: Running, Role: LEADER
I20260809 04:49:15.171747 29886 consensus_queue.cc:237] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28044201fe7c47928c6337921990acbc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42769 } } peers { permanent_uuid: "55bcc2348bcf458db7a30d04ee6db051" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 34867 } } peers { permanent_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36333 } }
I20260809 04:49:15.175077 29354 catalog_manager.cc:5495] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55bcc2348bcf458db7a30d04ee6db051 (127.7.171.3). New cstate: current_term: 1 leader_uuid: "55bcc2348bcf458db7a30d04ee6db051" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28044201fe7c47928c6337921990acbc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55bcc2348bcf458db7a30d04ee6db051" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 34867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36333 } health_report { overall_health: UNKNOWN } } }
W20260809 04:49:15.178403 29861 tablet.cc:2369] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:49:15.193475 29712 tablet.cc:2369] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:49:15.217499 29567 tablet.cc:2369] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:49:15.223486 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.219691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39174 (local address 127.7.171.3:34867)
0809 04:49:15.219823 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:15.219827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:15.219842 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:49:15.219922 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:15.219924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:15.220043 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:15.220198 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:15.220468 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.221013 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.221881 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.222136 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.222215 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.222492 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:15.222896 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:15.223053 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:15.223306 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 04:49:15.223375 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:49:15.229856 29506 raft_consensus.cc:1261] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267 [term 1 FOLLOWER]: Refusing update from remote peer 55bcc2348bcf458db7a30d04ee6db051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:15.229856 29659 raft_consensus.cc:1261] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc [term 1 FOLLOWER]: Refusing update from remote peer 55bcc2348bcf458db7a30d04ee6db051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:15.230773 29890 consensus_queue.cc:1035] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa85a3196a3e48dab1d8d44fde9c2267" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 36333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:15.231101 29886 consensus_queue.cc:1035] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28044201fe7c47928c6337921990acbc" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 42769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:15.238207 29893 mvcc.cc:205] Tried to move back new op lower bound from 7316483912613171200 to 7316483912386621440. Current Snapshot: MvccSnapshot[applied={T|T < 7316483912613171200}]
I20260809 04:49:15.238831 29894 mvcc.cc:205] Tried to move back new op lower bound from 7316483912613171200 to 7316483912386621440. Current Snapshot: MvccSnapshot[applied={T|T < 7316483912613171200}]
I20260809 04:49:15.241629 29895 mvcc.cc:205] Tried to move back new op lower bound from 7316483912613171200 to 7316483912386621440. Current Snapshot: MvccSnapshot[applied={T|T < 7316483912613171200}]
I20260809 04:49:15.247669 29356 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:15.247819 29356 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:15.250478 29356 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56deebd60bf7448ab2a22d95d064d7dc: Sending DeleteTablet for 3 replicas of tablet 2dc5740a1d7c4af7a455f8c230e71437
I20260809 04:49:15.251451 29783 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dc5740a1d7c4af7a455f8c230e71437 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:15 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:39170
I20260809 04:49:15.251416 29637 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dc5740a1d7c4af7a455f8c230e71437 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:15 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:50114
I20260809 04:49:15.251601 29490 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dc5740a1d7c4af7a455f8c230e71437 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:15 UTC) from {username='kudu', principal='kudu/127.7.171.62@KRBTEST.COM'} at 127.0.0.1:34284
I20260809 04:49:15.252219 29905 tablet_replica.cc:330] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267: stopping tablet replica
I20260809 04:49:15.252650 29903 tablet_replica.cc:330] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: stopping tablet replica
I20260809 04:49:15.252636 29905 raft_consensus.cc:2229] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:49:15.252676 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 29422
I20260809 04:49:15.252947 29903 raft_consensus.cc:2229] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:49:15.252985 29905 raft_consensus.cc:2258] T 2dc5740a1d7c4af7a455f8c230e71437 P fa85a3196a3e48dab1d8d44fde9c2267 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:15.253244 29903 raft_consensus.cc:2258] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:15.253636 29904 tablet_replica.cc:330] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: stopping tablet replica
I20260809 04:49:15.253865 29904 raft_consensus.cc:2229] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:49:15.254133 29904 raft_consensus.cc:2258] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:15.256260 29904 ts_tablet_manager.cc:1905] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:15.260100 29904 ts_tablet_manager.cc:1918] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:15.260212 29904 log.cc:1196] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/2dc5740a1d7c4af7a455f8c230e71437
I20260809 04:49:15.260594 29904 ts_tablet_manager.cc:1939] T 2dc5740a1d7c4af7a455f8c230e71437 P 55bcc2348bcf458db7a30d04ee6db051: Deleting consensus metadata
I20260809 04:49:15.261166 29903 ts_tablet_manager.cc:1905] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:15.261979 29342 catalog_manager.cc:4841] TS 55bcc2348bcf458db7a30d04ee6db051 (127.7.171.3:34867): tablet 2dc5740a1d7c4af7a455f8c230e71437 (table test-table [id=825b18569e67486d8e7599aa7add2e59]) successfully deleted
W20260809 04:49:15.263409 29342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:36333 (error 108)
I20260809 04:49:15.263882 29903 ts_tablet_manager.cc:1918] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:15.263985 29903 log.cc:1196] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/2dc5740a1d7c4af7a455f8c230e71437
I20260809 04:49:15.264338 29903 ts_tablet_manager.cc:1939] T 2dc5740a1d7c4af7a455f8c230e71437 P 28044201fe7c47928c6337921990acbc: Deleting consensus metadata
I20260809 04:49:15.264676 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.264352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:36333 (local address 127.0.0.1:34290)
0809 04:49:15.264497 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:15.264580 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:36333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 04:49:15.264698 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 29572
W20260809 04:49:15.267958 29342 catalog_manager.cc:4584] TS fa85a3196a3e48dab1d8d44fde9c2267 (127.7.171.1:36333): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2dc5740a1d7c4af7a455f8c230e71437: Network error: Client connection negotiation failed: client connection to 127.7.171.1:36333: connect: Connection refused (error 111)
I20260809 04:49:15.274765 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 29716
I20260809 04:49:15.275019 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.274690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:42769 (local address 127.0.0.1:50116)
0809 04:49:15.274864 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:15.274922 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.2:42769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 04:49:15.283492 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 29323
2026-08-09T04:49:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2569 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T04:49:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:15Z Disabled control of system clock
I20260809 04:49:15.319922 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:40019
--webserver_interface=127.7.171.62
--webserver_port=0
--builtin_ntp_servers=127.7.171.20:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:40019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:49:15.435591 29910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:15.435860 29910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:15.435896 29910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:15.439992 29910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:15.440083 29910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:15.440101 29910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:15.440119 29910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:15.440133 29910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:15.445073 29910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:40019
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:40019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:15.446280 29910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:15.447727 29910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:15.456578 29915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:15.456630 29918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:15.456583 29916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:15.456799 29910 server_base.cc:1034] running on GCE node
I20260809 04:49:15.457527 29910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:15.458596 29910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:15.459775 29910 hybrid_clock.cc:648] HybridClock initialized: now 1786250955459739 us; error 44 us; skew 500 ppm
I20260809 04:49:15.461804 29910 webserver.cc:460] Webserver started at http://127.7.171.62:41213/ using document root <none> and password file <none>
I20260809 04:49:15.462411 29910 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:15.462476 29910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:15.462700 29910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:15.464854 29910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "a23fb9f4d9dd46cd9d24cd85711b8beb"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1zqn"
server_key: "eb91641f1ef80a3a968c8bb797981779"
server_key_iv: "7de164df3a617bf18c6d1e376a044750"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.465456 29910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "a23fb9f4d9dd46cd9d24cd85711b8beb"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1zqn"
server_key: "eb91641f1ef80a3a968c8bb797981779"
server_key_iv: "7de164df3a617bf18c6d1e376a044750"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.469220 29910 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 04:49:15.471740 29924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:15.473031 29910 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:49:15.473173 29910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "a23fb9f4d9dd46cd9d24cd85711b8beb"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1zqn"
server_key: "eb91641f1ef80a3a968c8bb797981779"
server_key_iv: "7de164df3a617bf18c6d1e376a044750"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.473285 29910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:15.490314 29910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:15.491094 29910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:15.491312 29910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:15.498901 29910 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:40019
I20260809 04:49:15.498912 29982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:40019 every 8 connection(s)
I20260809 04:49:15.499904 29910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:15.503201 29983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:15.506372 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 29910
I20260809 04:49:15.506541 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:15.506892 7852 external_mini_cluster.cc:1428] Setting key c1bb4e3534d22010bca6a19dbdb23d53
I20260809 04:49:15.509670 29983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb: Bootstrap starting.
I20260809 04:49:15.511929 29983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:15.512737 29983 log.cc:824] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:15.514782 29983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb: No bootstrap required, opened a new log
I20260809 04:49:15.516381 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.508388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59192 (local address 127.7.171.62:40019)
0809 04:49:15.509090 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:15.509106 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:15.509141 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:49:15.509827 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:15.509846 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:15.510244 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:15.510578 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:15.511031 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.511973 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.512999 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.513470 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.513649 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.514527 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:15.514533 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:15.514547 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:15.514993 (+ 446us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:15.515071 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:15.515092 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:15.515198 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:15.515345 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 04:49:15.515607 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":128,"threads_started":1}
I20260809 04:49:15.519286 29983 raft_consensus.cc:348] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23fb9f4d9dd46cd9d24cd85711b8beb" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 40019 } }
I20260809 04:49:15.519529 29983 raft_consensus.cc:374] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:15.519608 29983 raft_consensus.cc:729] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23fb9f4d9dd46cd9d24cd85711b8beb, State: Initialized, Role: FOLLOWER
I20260809 04:49:15.520119 29983 consensus_queue.cc:260] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23fb9f4d9dd46cd9d24cd85711b8beb" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 40019 } }
I20260809 04:49:15.520277 29983 raft_consensus.cc:388] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:15.520342 29983 raft_consensus.cc:482] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:15.520437 29983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:15.521402 29983 raft_consensus.cc:504] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23fb9f4d9dd46cd9d24cd85711b8beb" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 40019 } }
I20260809 04:49:15.521740 29983 leader_election.cc:302] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a23fb9f4d9dd46cd9d24cd85711b8beb; no voters:
I20260809 04:49:15.522056 29983 leader_election.cc:288] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:15.522149 29988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:15.522451 29988 raft_consensus.cc:686] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [term 1 LEADER]: Becoming Leader. State: Replica: a23fb9f4d9dd46cd9d24cd85711b8beb, State: Running, Role: LEADER
I20260809 04:49:15.522796 29988 consensus_queue.cc:237] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23fb9f4d9dd46cd9d24cd85711b8beb" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 40019 } }
I20260809 04:49:15.523357 29983 sys_catalog.cc:564] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:15.524458 29989 sys_catalog.cc:455] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a23fb9f4d9dd46cd9d24cd85711b8beb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23fb9f4d9dd46cd9d24cd85711b8beb" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 40019 } } }
I20260809 04:49:15.524622 29989 sys_catalog.cc:458] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:15.524973 29990 sys_catalog.cc:455] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [sys.catalog]: SysCatalogTable state changed. Reason: New leader a23fb9f4d9dd46cd9d24cd85711b8beb. Latest consensus state: current_term: 1 leader_uuid: "a23fb9f4d9dd46cd9d24cd85711b8beb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23fb9f4d9dd46cd9d24cd85711b8beb" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 40019 } } }
I20260809 04:49:15.525103 29990 sys_catalog.cc:458] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb [sys.catalog]: This master's current role is: LEADER
W20260809 04:49:15.528010 30004 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:15.528129 30004 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:15.528214 29998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:15.529076 29998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:15.534874 29998 catalog_manager.cc:1347] Generated new cluster ID: 070245c760b7451c8d58b75b56c116ce
I20260809 04:49:15.534941 29998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:15.568003 29998 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:15.569082 29998 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:15.575558 29998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb: Generated new TSK 0
I20260809 04:49:15.576419 29998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:15.593600 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.1:0
--local_ip_for_outbound_sockets=127.7.171.1
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:40019
--builtin_ntp_servers=127.7.171.20:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:49:15.710304 30009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:15.710578 30009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:15.710618 30009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:15.714684 30009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:15.714747 30009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:15.714800 30009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.1
I20260809 04:49:15.719621 30009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.171.1
--webserver_port=0
--tserver_master_addrs=127.7.171.62:40019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.1
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:15.720772 30009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:15.722188 30009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:15.732368 30014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:15.732446 30009 server_base.cc:1034] running on GCE node
W20260809 04:49:15.732375 30015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:15.732375 30017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:15.733016 30009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:15.733686 30009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:15.734918 30009 hybrid_clock.cc:648] HybridClock initialized: now 1786250955734896 us; error 34 us; skew 500 ppm
I20260809 04:49:15.737120 30009 webserver.cc:460] Webserver started at http://127.7.171.1:43347/ using document root <none> and password file <none>
I20260809 04:49:15.737775 30009 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:15.737881 30009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:15.738135 30009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:15.740396 30009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/data/instance:
uuid: "cb04d7b222b3493c8b02c172020362ec"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1zqn"
server_key: "251c75e37b20266fc3985f55d6fe802e"
server_key_iv: "b36a51c989a34591e6d141f09e668862"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.741052 30009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance:
uuid: "cb04d7b222b3493c8b02c172020362ec"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1zqn"
server_key: "251c75e37b20266fc3985f55d6fe802e"
server_key_iv: "b36a51c989a34591e6d141f09e668862"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.745168 30009 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:49:15.747752 30023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:15.749096 30009 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:15.749243 30009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/wal
uuid: "cb04d7b222b3493c8b02c172020362ec"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1zqn"
server_key: "251c75e37b20266fc3985f55d6fe802e"
server_key_iv: "b36a51c989a34591e6d141f09e668862"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.749368 30009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:15.777201 30009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:15.778019 30009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:15.778230 30009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:15.778895 30009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:15.779894 30009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:15.779976 30009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:15.780081 30009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:15.780131 30009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:15.790352 30009 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.1:43983
I20260809 04:49:15.790372 30147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.1:43983 every 8 connection(s)
I20260809 04:49:15.791359 30009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/data/info.pb
I20260809 04:49:15.791936 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 30009
I20260809 04:49:15.792112 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-0/wal/instance
I20260809 04:49:15.792412 7852 external_mini_cluster.cc:1428] Setting key 0f365fc9510a0c45e9b2757ffcd4aa04
I20260809 04:49:15.794770 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.2:0
--local_ip_for_outbound_sockets=127.7.171.2
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:40019
--builtin_ntp_servers=127.7.171.20:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:49:15.801275 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.793420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.1:56237 (local address 127.7.171.62:40019)
0809 04:49:15.793583 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:15.793587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:15.794805 (+ 1218us) server_negotiation.cc:407] Connection header received
0809 04:49:15.796332 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:15.796336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:15.796396 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:15.796480 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:15.797411 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.797957 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.799152 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.799429 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.799541 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.800749 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:15.800753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:15.800756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:15.800771 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:15.800799 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:15.800806 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:15.800883 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:15.801057 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 04:49:15.801126 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:49:15.802176 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.793904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:40019 (local address 127.7.171.1:56237)
0809 04:49:15.794655 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:15.794697 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:15.796075 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:15.796665 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:15.796671 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:15.796760 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:15.797260 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:15.797273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.798096 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.798107 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:15.799014 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:15.799025 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.799604 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.799609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:15.799773 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.800457 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:49:15.800458 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:15.800499 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:49:15.800574 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:15.800898 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:15.800907 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:15.800913 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:49:15.801052 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 04:49:15.801343 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":165,"threads_started":1}
I20260809 04:49:15.803484 30148 heartbeater.cc:347] Connected to a master server at 127.7.171.62:40019
I20260809 04:49:15.803740 30148 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:15.804348 30148 heartbeater.cc:511] Master 127.7.171.62:40019 requested a full tablet report, sending...
I20260809 04:49:15.806134 29942 ts_manager.cc:194] Registered new tserver with Master: cb04d7b222b3493c8b02c172020362ec (127.7.171.1:43983)
I20260809 04:49:15.807762 29942 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.1:56237
W20260809 04:49:15.912086 30151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:15.912366 30151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:15.912405 30151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:15.916477 30151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:15.916533 30151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:15.916589 30151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.2
I20260809 04:49:15.921296 30151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.171.2
--webserver_port=0
--tserver_master_addrs=127.7.171.62:40019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.2
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:15.922367 30151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:15.923884 30151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:15.934057 30156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:15.934008 30157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:15.934008 30160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:15.934162 30151 server_base.cc:1034] running on GCE node
I20260809 04:49:15.934736 30151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:15.935346 30151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:15.936504 30151 hybrid_clock.cc:648] HybridClock initialized: now 1786250955936473 us; error 44 us; skew 500 ppm
I20260809 04:49:15.938364 30151 webserver.cc:460] Webserver started at http://127.7.171.2:39299/ using document root <none> and password file <none>
I20260809 04:49:15.938899 30151 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:15.939035 30151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:15.939218 30151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:15.941282 30151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/data/instance:
uuid: "5172961eef20420c8c639c0e0871021a"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1zqn"
server_key: "f7277b472fe08baf5b922f63dd48e67a"
server_key_iv: "42ea6fd40544b39c363588bb1eb7e22d"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.941845 30151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance:
uuid: "5172961eef20420c8c639c0e0871021a"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1zqn"
server_key: "f7277b472fe08baf5b922f63dd48e67a"
server_key_iv: "42ea6fd40544b39c363588bb1eb7e22d"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.945829 30151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:49:15.948328 30167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:15.949599 30151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:15.949720 30151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/wal
uuid: "5172961eef20420c8c639c0e0871021a"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1zqn"
server_key: "f7277b472fe08baf5b922f63dd48e67a"
server_key_iv: "42ea6fd40544b39c363588bb1eb7e22d"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.949829 30151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:15.968225 30151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:15.968910 30151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:15.969139 30151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:15.969744 30151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:15.970686 30151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:15.970736 30151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:15.970775 30151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:15.970824 30151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:15.980418 30151 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.2:39723
I20260809 04:49:15.980486 30287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.2:39723 every 8 connection(s)
I20260809 04:49:15.981519 30151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/data/info.pb
I20260809 04:49:15.982390 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 30151
I20260809 04:49:15.982519 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/wal/instance
I20260809 04:49:15.982831 7852 external_mini_cluster.cc:1428] Setting key dd0d516d05caa18571b80549f762cc50
I20260809 04:49:15.984905 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.171.3:0
--local_ip_for_outbound_sockets=127.7.171.3
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:40019
--builtin_ntp_servers=127.7.171.20:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:49:15.995061 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.983470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:58917 (local address 127.7.171.62:40019)
0809 04:49:15.983596 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:15.983607 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:15.987306 (+ 3699us) server_negotiation.cc:407] Connection header received
0809 04:49:15.988811 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:15.988815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:15.988875 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:15.988970 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:15.990159 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.991097 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.992573 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.992994 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.993121 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.994392 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:15.994408 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:15.994414 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:15.994431 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:15.994459 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:15.994472 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:15.994566 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:15.994768 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 04:49:15.994849 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 04:49:15.995841 30290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.983974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:40019 (local address 127.7.171.2:58917)
0809 04:49:15.987156 (+ 3182us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:15.987200 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:15.988599 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:15.989253 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:15.989261 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:15.989350 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:15.989982 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:15.989999 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.991267 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.991279 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:15.992429 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:15.992441 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.993175 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.993178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:15.993343 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.994098 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:49:15.994100 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:15.994145 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:49:15.994213 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:15.994596 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:15.994606 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:15.994611 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:49:15.994730 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 04:49:15.994993 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2775,"thread_start_us":148,"threads_started":1}
I20260809 04:49:15.997156 30288 heartbeater.cc:347] Connected to a master server at 127.7.171.62:40019
I20260809 04:49:15.997411 30288 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:15.997957 30288 heartbeater.cc:511] Master 127.7.171.62:40019 requested a full tablet report, sending...
I20260809 04:49:15.999027 29942 ts_manager.cc:194] Registered new tserver with Master: 5172961eef20420c8c639c0e0871021a (127.7.171.2:39723)
I20260809 04:49:15.999765 29942 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.2:58917
W20260809 04:49:16.103440 30291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:16.103704 30291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:16.103744 30291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:16.107736 30291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:16.107800 30291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:16.107854 30291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.171.3
I20260809 04:49:16.112707 30291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.171.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.171.3
--webserver_port=0
--tserver_master_addrs=127.7.171.62:40019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.171.3
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:16.113891 30291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:16.115273 30291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:16.125495 30301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:16.125495 30299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:16.125495 30297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:16.125851 30291 server_base.cc:1034] running on GCE node
I20260809 04:49:16.126298 30291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:16.127008 30291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:16.128181 30291 hybrid_clock.cc:648] HybridClock initialized: now 1786250956128169 us; error 29 us; skew 500 ppm
I20260809 04:49:16.129925 30291 webserver.cc:460] Webserver started at http://127.7.171.3:44161/ using document root <none> and password file <none>
I20260809 04:49:16.130580 30291 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:16.130681 30291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:16.130913 30291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:16.133164 30291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/data/instance:
uuid: "a6134596beb04bcaace5bf1d3df90464"
format_stamp: "Formatted at 2026-08-09 04:49:16 on dist-test-slave-1zqn"
server_key: "f325132e0f3efb4465cc834a467fad16"
server_key_iv: "407181696051ce8ff9c02664c7775312"
server_key_version: "encryptionkey@0"
I20260809 04:49:16.133731 30291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance:
uuid: "a6134596beb04bcaace5bf1d3df90464"
format_stamp: "Formatted at 2026-08-09 04:49:16 on dist-test-slave-1zqn"
server_key: "f325132e0f3efb4465cc834a467fad16"
server_key_iv: "407181696051ce8ff9c02664c7775312"
server_key_version: "encryptionkey@0"
I20260809 04:49:16.137368 30291 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:49:16.139770 30307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:16.140964 30291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:16.141105 30291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/wal
uuid: "a6134596beb04bcaace5bf1d3df90464"
format_stamp: "Formatted at 2026-08-09 04:49:16 on dist-test-slave-1zqn"
server_key: "f325132e0f3efb4465cc834a467fad16"
server_key_iv: "407181696051ce8ff9c02664c7775312"
server_key_version: "encryptionkey@0"
I20260809 04:49:16.141213 30291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:16.156951 30291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:16.157644 30291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:16.157848 30291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:16.158470 30291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:49:16.159646 30291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:49:16.159723 30291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:16.159792 30291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:49:16.159823 30291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:16.168854 30291 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.3:41847
I20260809 04:49:16.168884 30429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.3:41847 every 8 connection(s)
I20260809 04:49:16.169766 30291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/data/info.pb
I20260809 04:49:16.173564 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 30291
I20260809 04:49:16.173676 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/wal/instance
I20260809 04:49:16.173995 7852 external_mini_cluster.cc:1428] Setting key d90f39042514d16e4fe6a9606c55873c
I20260809 04:49:16.180826 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.171642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.3:54133 (local address 127.7.171.62:40019)
0809 04:49:16.171784 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:16.171787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:16.172985 (+ 1198us) server_negotiation.cc:407] Connection header received
0809 04:49:16.174974 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:16.174978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:16.175042 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:16.175140 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:16.176450 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.177152 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.178644 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.178897 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.178991 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.180307 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:16.180310 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:16.180313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:16.180327 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:16.180351 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:16.180360 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:16.180438 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:16.180613 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 04:49:16.180673 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:49:16.181763 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.172149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:40019 (local address 127.7.171.3:54133)
0809 04:49:16.172833 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:16.172874 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:16.174709 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:16.175380 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:16.175390 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:16.175509 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:16.176259 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.176278 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.177329 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.177332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.178466 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.178477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.179069 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.179072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.179229 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.179958 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:49:16.179960 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:16.180003 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:49:16.180122 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:16.180460 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:16.180467 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:16.180471 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:49:16.180567 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 04:49:16.180811 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":129,"threads_started":1}
I20260809 04:49:16.183004 30430 heartbeater.cc:347] Connected to a master server at 127.7.171.62:40019
I20260809 04:49:16.183300 30430 heartbeater.cc:464] Registering TS with master...
I20260809 04:49:16.183892 30430 heartbeater.cc:511] Master 127.7.171.62:40019 requested a full tablet report, sending...
I20260809 04:49:16.185040 29942 ts_manager.cc:194] Registered new tserver with Master: a6134596beb04bcaace5bf1d3df90464 (127.7.171.3:41847)
I20260809 04:49:16.185652 29942 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.171.3:54133
I20260809 04:49:16.188916 7852 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:49:16.194801 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.190945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59200 (local address 127.7.171.62:40019)
0809 04:49:16.191111 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:16.191118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:16.191273 (+ 155us) server_negotiation.cc:407] Connection header received
0809 04:49:16.191368 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:16.191370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:16.191411 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:16.191513 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:16.191730 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.192290 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.193161 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.193373 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.193453 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.194318 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:16.194321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:16.194323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:16.194335 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:16.194352 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:16.194360 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:16.194416 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:16.194544 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 04:49:16.194628 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 04:49:16.198015 29942 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:49:16.201113 29942 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:49:16.215996 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.210090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:41847 (local address 127.0.0.1:56602)
0809 04:49:16.211224 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:16.211238 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:16.211325 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:16.211955 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:16.211960 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:16.211976 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:16.212089 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.212098 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.213524 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.213527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.214622 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.214633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.215436 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.215439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.215551 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.215589 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 04:49:16.215710 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 04:49:16.215827 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"thread_start_us":95,"threads_started":1}
I20260809 04:49:16.216261 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.209970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39742 (local address 127.7.171.2:39723)
0809 04:49:16.211030 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:16.211044 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:16.211071 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:49:16.211151 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:16.211158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:16.211376 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:16.211545 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:16.212198 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.212949 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.214560 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.215378 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.215491 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.215689 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:16.215964 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 04:49:16.216092 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":943,"thread_start_us":155,"threads_started":1}
I20260809 04:49:16.216374 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.210549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.7.171.3:41847)
0809 04:49:16.211140 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:16.211150 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:16.211439 (+ 289us) server_negotiation.cc:407] Connection header received
0809 04:49:16.211525 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:16.211533 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:16.211790 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:16.211938 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:16.212188 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.213357 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.214755 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.215305 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.215434 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.215636 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:16.216110 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 04:49:16.216229 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":141,"threads_started":1}
I20260809 04:49:16.216617 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.209798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.2:39723 (local address 127.0.0.1:39742)
0809 04:49:16.210659 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:16.210712 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:16.210851 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:16.211727 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:16.211735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:16.211761 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:16.211994 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.212002 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.213072 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.213078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.214426 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.214437 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.215714 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.215718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.215807 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.215826 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:49:16.215899 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 04:49:16.215955 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":88,"spinlock_wait_cycles":6912,"thread_start_us":86,"threads_started":1}
I20260809 04:49:16.216100 30445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.209808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:43983 (local address 127.0.0.1:59328)
0809 04:49:16.210660 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:16.210712 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:16.210852 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:16.211772 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:16.211775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:16.211790 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:16.212006 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.212014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.213490 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.213494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.214659 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.214666 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.215665 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.215668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.215765 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.215795 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 04:49:16.215839 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 04:49:16.215928 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":118,"spinlock_wait_cycles":4352,"thread_start_us":101,"threads_started":1}
I20260809 04:49:16.217056 30449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.210666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59328 (local address 127.7.171.1:43983)
0809 04:49:16.211334 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:16.211345 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:16.211367 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:49:16.211451 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:16.211459 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:16.211646 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:16.211794 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:16.212134 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.213357 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.214789 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.215530 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.216211 (+ 681us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.216405 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:16.216472 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 04:49:16.216563 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":126,"threads_started":1}
I20260809 04:49:16.219555 30358 tablet_service.cc:1467] Processing CreateTablet for tablet b6acd72af03543ae83fccb9438d0e415 (DEFAULT_TABLE table=test-table [id=4e399c4152e8410dabf0daed2ab21032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:16.219650 30074 tablet_service.cc:1467] Processing CreateTablet for tablet b6acd72af03543ae83fccb9438d0e415 (DEFAULT_TABLE table=test-table [id=4e399c4152e8410dabf0daed2ab21032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:16.219641 30217 tablet_service.cc:1467] Processing CreateTablet for tablet b6acd72af03543ae83fccb9438d0e415 (DEFAULT_TABLE table=test-table [id=4e399c4152e8410dabf0daed2ab21032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:49:16.220649 30358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6acd72af03543ae83fccb9438d0e415. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:16.220705 30074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6acd72af03543ae83fccb9438d0e415. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:16.220705 30217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6acd72af03543ae83fccb9438d0e415. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:16.227608 30450 tablet_bootstrap.cc:492] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464: Bootstrap starting.
I20260809 04:49:16.228822 30452 tablet_bootstrap.cc:492] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec: Bootstrap starting.
I20260809 04:49:16.229914 30450 tablet_bootstrap.cc:654] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:16.230654 30452 tablet_bootstrap.cc:654] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:16.230682 30450 log.cc:824] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:16.231107 30451 tablet_bootstrap.cc:492] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a: Bootstrap starting.
I20260809 04:49:16.231545 30452 log.cc:824] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:16.232916 30450 tablet_bootstrap.cc:492] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464: No bootstrap required, opened a new log
I20260809 04:49:16.233130 30450 ts_tablet_manager.cc:1397] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 04:49:16.233407 30451 tablet_bootstrap.cc:654] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:16.234087 30451 log.cc:824] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:16.235847 30451 tablet_bootstrap.cc:492] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a: No bootstrap required, opened a new log
I20260809 04:49:16.236121 30451 ts_tablet_manager.cc:1397] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 04:49:16.236606 30450 raft_consensus.cc:348] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5172961eef20420c8c639c0e0871021a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 39723 } } peers { permanent_uuid: "cb04d7b222b3493c8b02c172020362ec" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 43983 } } peers { permanent_uuid: "a6134596beb04bcaace5bf1d3df90464" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41847 } }
I20260809 04:49:16.236848 30450 raft_consensus.cc:374] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:16.236915 30450 raft_consensus.cc:729] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6134596beb04bcaace5bf1d3df90464, State: Initialized, Role: FOLLOWER
I20260809 04:49:16.237376 30450 consensus_queue.cc:260] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5172961eef20420c8c639c0e0871021a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 39723 } } peers { permanent_uuid: "cb04d7b222b3493c8b02c172020362ec" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 43983 } } peers { permanent_uuid: "a6134596beb04bcaace5bf1d3df90464" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41847 } }
I20260809 04:49:16.238080 30430 heartbeater.cc:503] Master 127.7.171.62:40019 was elected leader, sending a full tablet report...
I20260809 04:49:16.238438 30450 ts_tablet_manager.cc:1428] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 04:49:16.240036 30451 raft_consensus.cc:348] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5172961eef20420c8c639c0e0871021a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 39723 } } peers { permanent_uuid: "cb04d7b222b3493c8b02c172020362ec" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 43983 } } peers { permanent_uuid: "a6134596beb04bcaace5bf1d3df90464" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41847 } }
I20260809 04:49:16.240334 30451 raft_consensus.cc:374] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:16.240401 30451 raft_consensus.cc:729] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5172961eef20420c8c639c0e0871021a, State: Initialized, Role: FOLLOWER
I20260809 04:49:16.240491 30452 tablet_bootstrap.cc:492] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec: No bootstrap required, opened a new log
I20260809 04:49:16.240720 30452 ts_tablet_manager.cc:1397] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260809 04:49:16.240904 30451 consensus_queue.cc:260] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5172961eef20420c8c639c0e0871021a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 39723 } } peers { permanent_uuid: "cb04d7b222b3493c8b02c172020362ec" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 43983 } } peers { permanent_uuid: "a6134596beb04bcaace5bf1d3df90464" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41847 } }
I20260809 04:49:16.241961 30451 ts_tablet_manager.cc:1428] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260809 04:49:16.242368 30288 heartbeater.cc:503] Master 127.7.171.62:40019 was elected leader, sending a full tablet report...
I20260809 04:49:16.245085 30452 raft_consensus.cc:348] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5172961eef20420c8c639c0e0871021a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 39723 } } peers { permanent_uuid: "cb04d7b222b3493c8b02c172020362ec" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 43983 } } peers { permanent_uuid: "a6134596beb04bcaace5bf1d3df90464" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41847 } }
I20260809 04:49:16.245448 30452 raft_consensus.cc:374] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:16.245559 30452 raft_consensus.cc:729] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb04d7b222b3493c8b02c172020362ec, State: Initialized, Role: FOLLOWER
I20260809 04:49:16.246088 30452 consensus_queue.cc:260] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5172961eef20420c8c639c0e0871021a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 39723 } } peers { permanent_uuid: "cb04d7b222b3493c8b02c172020362ec" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 43983 } } peers { permanent_uuid: "a6134596beb04bcaace5bf1d3df90464" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41847 } }
I20260809 04:49:16.247241 30452 ts_tablet_manager.cc:1428] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 04:49:16.248250 30148 heartbeater.cc:503] Master 127.7.171.62:40019 was elected leader, sending a full tablet report...
W20260809 04:49:16.292757 30149 tablet.cc:2369] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:49:16.310382 30458 raft_consensus.cc:482] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:49:16.310632 30458 raft_consensus.cc:504] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5172961eef20420c8c639c0e0871021a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 39723 } } peers { permanent_uuid: "cb04d7b222b3493c8b02c172020362ec" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 43983 } } peers { permanent_uuid: "a6134596beb04bcaace5bf1d3df90464" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41847 } }
I20260809 04:49:16.311908 30458 leader_election.cc:288] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb04d7b222b3493c8b02c172020362ec (127.7.171.1:43983), a6134596beb04bcaace5bf1d3df90464 (127.7.171.3:41847)
I20260809 04:49:16.316023 30290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.312137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:43983 (local address 127.7.171.2:46149)
0809 04:49:16.312290 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:16.312307 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:16.312405 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:16.312711 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:16.312714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:16.312731 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:16.312851 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.312857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.313597 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.313602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.314733 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.314745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.315598 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.315601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.315700 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.315725 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:49:16.315807 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 04:49:16.315871 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":55}
I20260809 04:49:16.316028 30449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.312353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:46149 (local address 127.7.171.1:43983)
0809 04:49:16.312487 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:16.312494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:16.312510 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:49:16.312569 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:16.312571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:16.312611 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:16.312699 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:16.312978 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.313483 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.314891 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.315480 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.315566 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.315618 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:16.315856 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 04:49:16.315923 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:49:16.316676 30461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.312233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.3:41847 (local address 127.7.171.2:36079)
0809 04:49:16.312660 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:16.312672 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:16.312798 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:16.313074 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:16.313077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:16.313093 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:49:16.313225 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.313231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.313982 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.313986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.315315 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:16.315326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.316321 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.316325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:16.316410 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.316443 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 04:49:16.316523 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 04:49:16.316571 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":3,"thread_start_us":166,"threads_started":1}
I20260809 04:49:16.316740 30448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.312338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.171.2:36079 (local address 127.7.171.3:41847)
0809 04:49:16.312473 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:16.312478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:16.312763 (+ 285us) server_negotiation.cc:407] Connection header received
0809 04:49:16.312925 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:16.312934 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:16.312977 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:16.313063 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:49:16.313340 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.313847 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.315454 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.316037 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.316165 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.316237 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:16.316558 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 04:49:16.316631 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:49:16.316744 30096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6acd72af03543ae83fccb9438d0e415" candidate_uuid: "5172961eef20420c8c639c0e0871021a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb04d7b222b3493c8b02c172020362ec" is_pre_election: true
I20260809 04:49:16.317065 30096 raft_consensus.cc:2454] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5172961eef20420c8c639c0e0871021a in term 0.
I20260809 04:49:16.317360 30380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6acd72af03543ae83fccb9438d0e415" candidate_uuid: "5172961eef20420c8c639c0e0871021a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6134596beb04bcaace5bf1d3df90464" is_pre_election: true
I20260809 04:49:16.317602 30169 leader_election.cc:302] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5172961eef20420c8c639c0e0871021a, cb04d7b222b3493c8b02c172020362ec; no voters:
I20260809 04:49:16.317637 30380 raft_consensus.cc:2454] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5172961eef20420c8c639c0e0871021a in term 0.
I20260809 04:49:16.317866 30458 raft_consensus.cc:2781] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:49:16.317956 30458 raft_consensus.cc:482] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:49:16.318010 30458 raft_consensus.cc:3037] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:16.319068 30458 raft_consensus.cc:504] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5172961eef20420c8c639c0e0871021a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 39723 } } peers { permanent_uuid: "cb04d7b222b3493c8b02c172020362ec" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 43983 } } peers { permanent_uuid: "a6134596beb04bcaace5bf1d3df90464" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41847 } }
I20260809 04:49:16.319461 30458 leader_election.cc:288] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [CANDIDATE]: Term 1 election: Requested vote from peers cb04d7b222b3493c8b02c172020362ec (127.7.171.1:43983), a6134596beb04bcaace5bf1d3df90464 (127.7.171.3:41847)
I20260809 04:49:16.319823 30380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6acd72af03543ae83fccb9438d0e415" candidate_uuid: "5172961eef20420c8c639c0e0871021a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6134596beb04bcaace5bf1d3df90464"
I20260809 04:49:16.319834 30096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6acd72af03543ae83fccb9438d0e415" candidate_uuid: "5172961eef20420c8c639c0e0871021a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb04d7b222b3493c8b02c172020362ec"
I20260809 04:49:16.319924 30380 raft_consensus.cc:3037] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:16.319965 30096 raft_consensus.cc:3037] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:16.321013 30380 raft_consensus.cc:2454] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5172961eef20420c8c639c0e0871021a in term 1.
I20260809 04:49:16.321049 30096 raft_consensus.cc:2454] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5172961eef20420c8c639c0e0871021a in term 1.
I20260809 04:49:16.321310 30171 leader_election.cc:302] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5172961eef20420c8c639c0e0871021a, a6134596beb04bcaace5bf1d3df90464; no voters:
I20260809 04:49:16.321496 30458 raft_consensus.cc:2781] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:16.321722 30458 raft_consensus.cc:686] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 1 LEADER]: Becoming Leader. State: Replica: 5172961eef20420c8c639c0e0871021a, State: Running, Role: LEADER
I20260809 04:49:16.322077 30458 consensus_queue.cc:237] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5172961eef20420c8c639c0e0871021a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 39723 } } peers { permanent_uuid: "cb04d7b222b3493c8b02c172020362ec" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 43983 } } peers { permanent_uuid: "a6134596beb04bcaace5bf1d3df90464" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41847 } }
I20260809 04:49:16.325430 29942 catalog_manager.cc:5495] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5172961eef20420c8c639c0e0871021a (127.7.171.2). New cstate: current_term: 1 leader_uuid: "5172961eef20420c8c639c0e0871021a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5172961eef20420c8c639c0e0871021a" member_type: VOTER last_known_addr { host: "127.7.171.2" port: 39723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb04d7b222b3493c8b02c172020362ec" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 43983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6134596beb04bcaace5bf1d3df90464" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41847 } health_report { overall_health: UNKNOWN } } }
I20260809 04:49:16.340528 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.336843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39748 (local address 127.7.171.2:39723)
0809 04:49:16.336985 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:16.336990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:16.337005 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:49:16.337188 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:16.337190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:16.337247 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:16.337358 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:16.337605 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.338104 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.339009 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:16.339227 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:16.339334 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:16.339655 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:16.340149 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:16.340279 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:16.340319 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 04:49:16.340370 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:49:16.346618 30380 raft_consensus.cc:1261] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464 [term 1 FOLLOWER]: Refusing update from remote peer 5172961eef20420c8c639c0e0871021a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:16.346674 30096 raft_consensus.cc:1261] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec [term 1 FOLLOWER]: Refusing update from remote peer 5172961eef20420c8c639c0e0871021a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:16.347476 30462 consensus_queue.cc:1035] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb04d7b222b3493c8b02c172020362ec" member_type: VOTER last_known_addr { host: "127.7.171.1" port: 43983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:16.347682 30458 consensus_queue.cc:1035] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6134596beb04bcaace5bf1d3df90464" member_type: VOTER last_known_addr { host: "127.7.171.3" port: 41847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:16.356818 30467 mvcc.cc:205] Tried to move back new op lower bound from 7316483917188595712 to 7316483917097705472. Current Snapshot: MvccSnapshot[applied={T|T < 7316483917188595712}]
I20260809 04:49:16.357153 30468 mvcc.cc:205] Tried to move back new op lower bound from 7316483917188595712 to 7316483917097705472. Current Snapshot: MvccSnapshot[applied={T|T < 7316483917188595712}]
I20260809 04:49:16.365919 29940 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:16.366106 29940 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:49:16.368531 29940 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a23fb9f4d9dd46cd9d24cd85711b8beb: Sending DeleteTablet for 3 replicas of tablet b6acd72af03543ae83fccb9438d0e415
I20260809 04:49:16.369349 30074 tablet_service.cc:1514] Processing DeleteTablet for tablet b6acd72af03543ae83fccb9438d0e415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:16 UTC) from {username='slave'} at 127.0.0.1:59328
I20260809 04:49:16.369505 30358 tablet_service.cc:1514] Processing DeleteTablet for tablet b6acd72af03543ae83fccb9438d0e415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:16 UTC) from {username='slave'} at 127.0.0.1:56602
I20260809 04:49:16.369650 30217 tablet_service.cc:1514] Processing DeleteTablet for tablet b6acd72af03543ae83fccb9438d0e415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:49:16 UTC) from {username='slave'} at 127.0.0.1:39742
I20260809 04:49:16.369956 30478 tablet_replica.cc:330] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464: stopping tablet replica
I20260809 04:49:16.370157 30479 tablet_replica.cc:330] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a: stopping tablet replica
I20260809 04:49:16.370343 30478 raft_consensus.cc:2229] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:49:16.370536 30479 raft_consensus.cc:2229] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:49:16.370662 30477 tablet_replica.cc:330] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec: stopping tablet replica
I20260809 04:49:16.370791 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 30009
I20260809 04:49:16.371011 30477 raft_consensus.cc:2229] T b6acd72af03543ae83fccb9438d0e415 P cb04d7b222b3493c8b02c172020362ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:49:16.371071 30479 raft_consensus.cc:2258] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:16.372648 30479 ts_tablet_manager.cc:1905] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:16.374502 30478 raft_consensus.cc:2258] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:49:16.376278 30479 ts_tablet_manager.cc:1918] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:16.376425 30479 log.cc:1196] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-1/wal/wals/b6acd72af03543ae83fccb9438d0e415
I20260809 04:49:16.376837 30479 ts_tablet_manager.cc:1939] T b6acd72af03543ae83fccb9438d0e415 P 5172961eef20420c8c639c0e0871021a: Deleting consensus metadata
I20260809 04:49:16.377306 30478 ts_tablet_manager.cc:1905] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:49:16.378269 29925 catalog_manager.cc:4841] TS 5172961eef20420c8c639c0e0871021a (127.7.171.2:39723): tablet b6acd72af03543ae83fccb9438d0e415 (table test-table [id=4e399c4152e8410dabf0daed2ab21032]) successfully deleted
I20260809 04:49:16.380465 30478 ts_tablet_manager.cc:1918] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:49:16.380570 30478 log.cc:1196] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464: Deleting WAL directory at /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250697618759-7852-0/minicluster-data/ts-2/wal/wals/b6acd72af03543ae83fccb9438d0e415
I20260809 04:49:16.380947 30478 ts_tablet_manager.cc:1939] T b6acd72af03543ae83fccb9438d0e415 P a6134596beb04bcaace5bf1d3df90464: Deleting consensus metadata
I20260809 04:49:16.381865 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 30151
W20260809 04:49:16.381865 29926 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.171.1:43983 (error 108)
I20260809 04:49:16.386181 29929 catalog_manager.cc:4841] TS a6134596beb04bcaace5bf1d3df90464 (127.7.171.3:41847): tablet b6acd72af03543ae83fccb9438d0e415 (table test-table [id=4e399c4152e8410dabf0daed2ab21032]) successfully deleted
I20260809 04:49:16.386893 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:16.386526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.1:43983 (local address 127.0.0.1:59336)
0809 04:49:16.386690 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:16.386781 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.1:43983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:49:16.387111 29926 catalog_manager.cc:4584] TS cb04d7b222b3493c8b02c172020362ec (127.7.171.1:43983): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6acd72af03543ae83fccb9438d0e415: Network error: Client connection negotiation failed: client connection to 127.7.171.1:43983: connect: Connection refused (error 111)
I20260809 04:49:16.391589 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 30291
I20260809 04:49:16.398834 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 29910
2026-08-09T04:49:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1111 ms)
[----------] 6 tests from EncryptionPolicyTest (10979 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:16 dist-test-slave-1zqn krb5kdc[30484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:16 dist-test-slave-1zqn krb5kdc[30484](info): set up 2 sockets
Aug 09 04:49:16 dist-test-slave-1zqn krb5kdc[30484](info): commencing operation
krb5kdc: starting...
W20260809 04:49:17.536319 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:17 dist-test-slave-1zqn krb5kdc[30484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:17.676625 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37817
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:17.789049 30498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:17.789382 30498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:17.789472 30498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:17.793454 30498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:17.793545 30498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:17.793586 30498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:17.793644 30498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:17.793684 30498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:17.798959 30498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37817
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:17.800036 30498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:17.801092 30498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:17.810427 30504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:17.810377 30503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:17.810400 30506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:17.810595 30498 server_base.cc:1034] running on GCE node
I20260809 04:49:17.811281 30498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:17.812577 30498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:17.813771 30498 hybrid_clock.cc:648] HybridClock initialized: now 1786250957813712 us; error 75 us; skew 500 ppm
Aug 09 04:49:17 dist-test-slave-1zqn krb5kdc[30484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250957, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:17.820667 30498 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:17.821621 30498 webserver.cc:460] Webserver started at http://127.0.0.1:34201/ using document root <none> and password file <none>
I20260809 04:49:17.822155 30498 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:17.822242 30498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:17.822487 30498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:17.824635 30498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "27ab3e82e6f84ac2bc1283a967f7f9a5"
format_stamp: "Formatted at 2026-08-09 04:49:17 on dist-test-slave-1zqn"
server_key: "923c3bcd83820105df858803f9171445"
server_key_iv: "d10d17ccfe9cba24de61cdb43dce084f"
server_key_version: "encryptionkey@0"
I20260809 04:49:17.825153 30498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "27ab3e82e6f84ac2bc1283a967f7f9a5"
format_stamp: "Formatted at 2026-08-09 04:49:17 on dist-test-slave-1zqn"
server_key: "923c3bcd83820105df858803f9171445"
server_key_iv: "d10d17ccfe9cba24de61cdb43dce084f"
server_key_version: "encryptionkey@0"
I20260809 04:49:17.829020 30498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:49:17.831559 30513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:17.833061 30498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:17.833179 30498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "27ab3e82e6f84ac2bc1283a967f7f9a5"
format_stamp: "Formatted at 2026-08-09 04:49:17 on dist-test-slave-1zqn"
server_key: "923c3bcd83820105df858803f9171445"
server_key_iv: "d10d17ccfe9cba24de61cdb43dce084f"
server_key_version: "encryptionkey@0"
I20260809 04:49:17.833280 30498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:17.860237 30498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:17.863261 30498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:17.863430 30498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:17.870888 30498 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37817
I20260809 04:49:17.870920 30570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37817 every 8 connection(s)
I20260809 04:49:17.871871 30498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:17.874258 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 30498
I20260809 04:49:17.874408 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:17.874715 7852 external_mini_cluster.cc:1428] Setting key b81611e7a9a82b2ff5afa229d33d3e6f
I20260809 04:49:17.875229 30571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:17.882038 30571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5: Bootstrap starting.
Aug 09 04:49:17 dist-test-slave-1zqn krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:49:17.884681 30571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:17.885725 30571 log.cc:824] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:17.888458 30571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5: No bootstrap required, opened a new log
I20260809 04:49:17.889721 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:17.876282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39639 (local address 127.0.0.1:37817)
0809 04:49:17.876783 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:17.876799 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:17.876836 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:49:17.877501 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:17.877517 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:17.877868 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:17.878233 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:17.879078 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:17.880163 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:17.881032 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:17.881515 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:17.881706 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:17.884416 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:17.884421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:17.884435 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:17.884464 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:17.886444 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:17.887111 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:17.887117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:17.887121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:17.887229 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:17.887594 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:17.887596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:17.887597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:17.888016 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:17.888244 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:17.888710 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 04:49:17.888987 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":167,"threads_started":1}
I20260809 04:49:17.892812 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:17.892870 30571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ab3e82e6f84ac2bc1283a967f7f9a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37817 } }
I20260809 04:49:17.893253 30571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:17.893339 30571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27ab3e82e6f84ac2bc1283a967f7f9a5, State: Initialized, Role: FOLLOWER
I20260809 04:49:17.893831 30571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ab3e82e6f84ac2bc1283a967f7f9a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37817 } }
I20260809 04:49:17.894032 30571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:17.894133 30571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:17.894240 30571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:17.895468 30571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ab3e82e6f84ac2bc1283a967f7f9a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37817 } }
I20260809 04:49:17.895906 30571 leader_election.cc:302] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27ab3e82e6f84ac2bc1283a967f7f9a5; no voters:
I20260809 04:49:17.896292 30571 leader_election.cc:288] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:17.896409 30581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:17.896662 30581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [term 1 LEADER]: Becoming Leader. State: Replica: 27ab3e82e6f84ac2bc1283a967f7f9a5, State: Running, Role: LEADER
I20260809 04:49:17.897079 30581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ab3e82e6f84ac2bc1283a967f7f9a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37817 } }
I20260809 04:49:17.897686 30571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:17.898313 30581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27ab3e82e6f84ac2bc1283a967f7f9a5. Latest consensus state: current_term: 1 leader_uuid: "27ab3e82e6f84ac2bc1283a967f7f9a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ab3e82e6f84ac2bc1283a967f7f9a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37817 } } }
I20260809 04:49:17.899127 30581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:17.899010 30583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27ab3e82e6f84ac2bc1283a967f7f9a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ab3e82e6f84ac2bc1283a967f7f9a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37817 } } }
I20260809 04:49:17.899500 30583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:17.899935 30593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:17.903165 30593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:17.905429 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:17.895276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41175 (local address 127.0.0.1:37817)
0809 04:49:17.895404 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:17.895409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:17.895539 (+ 130us) server_negotiation.cc:407] Connection header received
0809 04:49:17.895653 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:17.895657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:17.895707 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:17.895803 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:17.896659 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:17.897402 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:17.898560 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:17.898980 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:17.899062 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:17.901125 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:17.901129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:17.901133 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:17.901163 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:17.903148 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:17.903807 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:17.903811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:17.903813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:17.903912 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:17.904339 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:17.904343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:17.904345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:17.904603 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:17.904767 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:17.905122 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 04:49:17.905233 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 04:49:17.909260 30593 catalog_manager.cc:1347] Generated new cluster ID: 831d1f2ce8184b43b65d10c17fd6fedf
I20260809 04:49:17.909333 30593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:17.927068 30593 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:17.928560 30593 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:17.940217 30593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27ab3e82e6f84ac2bc1283a967f7f9a5: Generated new TSK 0
I20260809 04:49:17.941133 30593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:17.952169 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 30498
W20260809 04:49:17.961580 30482 connection.cc:535] client connection to 127.0.0.1:37817 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37817): Connection reset by peer (error 104)
2026-08-09T04:49:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1554 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:18 dist-test-slave-1zqn krb5kdc[30607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:18 dist-test-slave-1zqn krb5kdc[30607](info): set up 2 sockets
Aug 09 04:49:18 dist-test-slave-1zqn krb5kdc[30607](info): commencing operation
krb5kdc: starting...
W20260809 04:49:19.059335 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:19 dist-test-slave-1zqn krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:19.196478 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40289
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:19.310799 30621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:19.311115 30621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:19.311213 30621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:19.315218 30621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:19.315316 30621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:19.315354 30621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:19.315393 30621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:19.315428 30621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:19.320775 30621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40289
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:19.321832 30621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:19.322871 30621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:19.331393 30627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:19.331393 30626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:19.331452 30629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:19.331714 30621 server_base.cc:1034] running on GCE node
I20260809 04:49:19.332327 30621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:19.333508 30621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:19.334673 30621 hybrid_clock.cc:648] HybridClock initialized: now 1786250959334642 us; error 44 us; skew 500 ppm
Aug 09 04:49:19 dist-test-slave-1zqn krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250959, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:19.337626 30621 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:19.338466 30621 webserver.cc:460] Webserver started at http://127.0.0.1:33857/ using document root <none> and password file <none>
I20260809 04:49:19.339006 30621 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:19.339066 30621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:19.339226 30621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:19.341377 30621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "41b489f141d5413d98c104d931f4ba26"
format_stamp: "Formatted at 2026-08-09 04:49:19 on dist-test-slave-1zqn"
server_key: "ea1517f87f1cc905fee9db6c2a6ea1ac"
server_key_iv: "6bf46526e6ca8b2cfbd940a2aa7eed3c"
server_key_version: "encryptionkey@0"
I20260809 04:49:19.341893 30621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "41b489f141d5413d98c104d931f4ba26"
format_stamp: "Formatted at 2026-08-09 04:49:19 on dist-test-slave-1zqn"
server_key: "ea1517f87f1cc905fee9db6c2a6ea1ac"
server_key_iv: "6bf46526e6ca8b2cfbd940a2aa7eed3c"
server_key_version: "encryptionkey@0"
I20260809 04:49:19.345597 30621 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:49:19.347931 30637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:19.349164 30621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:19.349295 30621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "41b489f141d5413d98c104d931f4ba26"
format_stamp: "Formatted at 2026-08-09 04:49:19 on dist-test-slave-1zqn"
server_key: "ea1517f87f1cc905fee9db6c2a6ea1ac"
server_key_iv: "6bf46526e6ca8b2cfbd940a2aa7eed3c"
server_key_version: "encryptionkey@0"
I20260809 04:49:19.349399 30621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:19.360808 30621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:19.363890 30621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:19.364176 30621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:19.372118 30621 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40289
I20260809 04:49:19.372174 30697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40289 every 8 connection(s)
I20260809 04:49:19.373167 30621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:19.373709 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 30621
I20260809 04:49:19.373888 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:19.374156 7852 external_mini_cluster.cc:1428] Setting key c03f3dd25536e32fd4c3f14600448b86
I20260809 04:49:19.376533 30698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:19.383251 30698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26: Bootstrap starting.
I20260809 04:49:19.383507 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:19.375557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33819 (local address 127.0.0.1:40289)
0809 04:49:19.375976 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:19.375991 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:19.376034 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:49:19.376782 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:19.376801 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:19.377139 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:19.377414 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:19.377837 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:19.378880 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:19.380184 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:19.380648 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:19.380766 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:19.381742 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:19.381747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:19.381760 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:19.382136 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:19.382189 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:19.382212 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:19.382312 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:19.382546 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 04:49:19.382696 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":154,"threads_started":1}
I20260809 04:49:19.385950 30698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:19.386525 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:19.386816 30698 log.cc:824] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:19.389140 30698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26: No bootstrap required, opened a new log
I20260809 04:49:19.392715 30698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b489f141d5413d98c104d931f4ba26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40289 } }
I20260809 04:49:19.392933 30698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:19.393028 30698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41b489f141d5413d98c104d931f4ba26, State: Initialized, Role: FOLLOWER
I20260809 04:49:19.393523 30698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b489f141d5413d98c104d931f4ba26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40289 } }
I20260809 04:49:19.393679 30698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:19.393760 30698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:19.393909 30698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:19.394876 30698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b489f141d5413d98c104d931f4ba26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40289 } }
I20260809 04:49:19.395226 30698 leader_election.cc:302] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41b489f141d5413d98c104d931f4ba26; no voters:
I20260809 04:49:19.395550 30698 leader_election.cc:288] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:19.395655 30709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:19.395864 30709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [term 1 LEADER]: Becoming Leader. State: Replica: 41b489f141d5413d98c104d931f4ba26, State: Running, Role: LEADER
I20260809 04:49:19.396425 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:19.388953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35717 (local address 127.0.0.1:40289)
0809 04:49:19.389092 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:19.389097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:19.392339 (+ 3242us) server_negotiation.cc:407] Connection header received
0809 04:49:19.392480 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:19.392483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:19.392541 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:19.392626 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:19.392896 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:19.393494 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:19.394530 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:19.394755 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:19.394867 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:19.395781 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:19.395784 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:19.395787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:19.395802 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:19.395834 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:19.395851 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:19.395919 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:19.396215 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 04:49:19.396251 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:49:19.396286 30709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b489f141d5413d98c104d931f4ba26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40289 } }
I20260809 04:49:19.397022 30698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:19.398317 30710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41b489f141d5413d98c104d931f4ba26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b489f141d5413d98c104d931f4ba26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40289 } } }
I20260809 04:49:19.398452 30710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:19.398788 30711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41b489f141d5413d98c104d931f4ba26. Latest consensus state: current_term: 1 leader_uuid: "41b489f141d5413d98c104d931f4ba26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41b489f141d5413d98c104d931f4ba26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40289 } } }
I20260809 04:49:19.398869 30711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:19.398905 30718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:19.402021 30718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:19.408102 30718 catalog_manager.cc:1347] Generated new cluster ID: 6d357a59c49d4ffc9e2d91fe46188384
I20260809 04:49:19.408169 30718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:19.414404 30718 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:19.415604 30718 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:19.420998 30718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41b489f141d5413d98c104d931f4ba26: Generated new TSK 0
I20260809 04:49:19.421628 30718 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:19.430142 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 30621
W20260809 04:49:19.438458 30605 connection.cc:535] client connection to 127.0.0.1:40289 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40289): Connection reset by peer (error 104)
2026-08-09T04:49:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1476 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:19 dist-test-slave-1zqn krb5kdc[30732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:19 dist-test-slave-1zqn krb5kdc[30732](info): set up 2 sockets
Aug 09 04:49:19 dist-test-slave-1zqn krb5kdc[30732](info): commencing operation
krb5kdc: starting...
W20260809 04:49:20.575625 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:20 dist-test-slave-1zqn krb5kdc[30732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:20.715902 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:20.829566 30747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:20.829847 30747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:20.829888 30747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:20.833937 30747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:20.833997 30747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:20.834017 30747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:20.834033 30747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:20.834048 30747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:20.839619 30747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33693
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:20.840745 30747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:20.841809 30747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:20.850461 30756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:20.850471 30753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:20.850472 30754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:20.850809 30747 server_base.cc:1034] running on GCE node
I20260809 04:49:20.851472 30747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:20.852730 30747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:20.853952 30747 hybrid_clock.cc:648] HybridClock initialized: now 1786250960853943 us; error 49 us; skew 500 ppm
Aug 09 04:49:20 dist-test-slave-1zqn krb5kdc[30732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250960, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:20.856972 30747 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:20.857890 30747 webserver.cc:460] Webserver started at http://127.0.0.1:39499/ using document root <none> and password file <none>
I20260809 04:49:20.858455 30747 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:20.858522 30747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:20.858707 30747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:20.860800 30747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "06a902ca1bee43cca89a65adfcd5fe81"
format_stamp: "Formatted at 2026-08-09 04:49:20 on dist-test-slave-1zqn"
server_key: "6778f7b00e345e5940daba936eb4a5c8"
server_key_iv: "aafb5660ccef5bca1cee844617be0f49"
server_key_version: "encryptionkey@0"
I20260809 04:49:20.861317 30747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "06a902ca1bee43cca89a65adfcd5fe81"
format_stamp: "Formatted at 2026-08-09 04:49:20 on dist-test-slave-1zqn"
server_key: "6778f7b00e345e5940daba936eb4a5c8"
server_key_iv: "aafb5660ccef5bca1cee844617be0f49"
server_key_version: "encryptionkey@0"
I20260809 04:49:20.865069 30747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:49:20.867478 30764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:20.868863 30747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:49:20.868976 30747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "06a902ca1bee43cca89a65adfcd5fe81"
format_stamp: "Formatted at 2026-08-09 04:49:20 on dist-test-slave-1zqn"
server_key: "6778f7b00e345e5940daba936eb4a5c8"
server_key_iv: "aafb5660ccef5bca1cee844617be0f49"
server_key_version: "encryptionkey@0"
I20260809 04:49:20.869110 30747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:20.873799 30747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:20.876891 30747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:20.877082 30747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:20.884528 30747 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33693
I20260809 04:49:20.884632 30818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33693 every 8 connection(s)
I20260809 04:49:20.885530 30747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:20.888443 30819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:20.893723 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 30747
I20260809 04:49:20.893868 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:20.894160 7852 external_mini_cluster.cc:1428] Setting key 4d52dd9a241e74736af090b9449e8fe2
I20260809 04:49:20.894284 30819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81: Bootstrap starting.
I20260809 04:49:20.896919 30819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:20.897859 30819 log.cc:824] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:20.899417 30822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:20.895521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35161 (local address 127.0.0.1:33693)
0809 04:49:20.895964 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:20.895980 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:20.896017 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:49:20.896676 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:20.896693 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:20.897014 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:20.897332 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:20.898173 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:20.898177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:20.898189 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:20.898199 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:20.898257 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:20.898277 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:20.898404 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:20.898524 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 04:49:20.898710 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":152,"threads_started":1}
I20260809 04:49:20.900146 30819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81: No bootstrap required, opened a new log
I20260809 04:49:20.902789 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:20.904878 30819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a902ca1bee43cca89a65adfcd5fe81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260809 04:49:20.905146 30819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:20.905185 30819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06a902ca1bee43cca89a65adfcd5fe81, State: Initialized, Role: FOLLOWER
I20260809 04:49:20.905706 30819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a902ca1bee43cca89a65adfcd5fe81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260809 04:49:20.905874 30819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:20.905962 30819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:20.906092 30819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:20.907267 30819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a902ca1bee43cca89a65adfcd5fe81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260809 04:49:20.907555 30822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:20.905442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45161 (local address 127.0.0.1:33693)
0809 04:49:20.905560 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:20.905564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:20.905807 (+ 243us) server_negotiation.cc:407] Connection header received
0809 04:49:20.906056 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:20.906060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:20.906103 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:20.906195 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:20.907056 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:20.907060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:20.907067 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:20.907087 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:20.907112 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:20.907123 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:20.907223 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:20.907327 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 04:49:20.907369 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:49:20.907658 30819 leader_election.cc:302] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06a902ca1bee43cca89a65adfcd5fe81; no voters:
I20260809 04:49:20.908035 30819 leader_election.cc:288] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:20.908175 30829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:20.908416 30829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [term 1 LEADER]: Becoming Leader. State: Replica: 06a902ca1bee43cca89a65adfcd5fe81, State: Running, Role: LEADER
I20260809 04:49:20.908789 30829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a902ca1bee43cca89a65adfcd5fe81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260809 04:49:20.909358 30819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:20.910574 30831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06a902ca1bee43cca89a65adfcd5fe81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a902ca1bee43cca89a65adfcd5fe81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } } }
I20260809 04:49:20.910712 30831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:20.911116 30841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:20.911072 30832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06a902ca1bee43cca89a65adfcd5fe81. Latest consensus state: current_term: 1 leader_uuid: "06a902ca1bee43cca89a65adfcd5fe81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06a902ca1bee43cca89a65adfcd5fe81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } } }
I20260809 04:49:20.911232 30832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:20.914392 30841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:20.920217 30841 catalog_manager.cc:1347] Generated new cluster ID: 476854cf41cb494fa2126adff95ac182
I20260809 04:49:20.920295 30841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:20.931604 30841 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:20.931948 30841 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:20.935935 30841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06a902ca1bee43cca89a65adfcd5fe81: Generated new TSK 0
I20260809 04:49:20.936599 30841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:20.945674 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 30747
2026-08-09T04:49:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:21 dist-test-slave-1zqn krb5kdc[30854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:21 dist-test-slave-1zqn krb5kdc[30854](info): set up 2 sockets
Aug 09 04:49:21 dist-test-slave-1zqn krb5kdc[30854](info): commencing operation
krb5kdc: starting...
W20260809 04:49:22.086107 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:22 dist-test-slave-1zqn krb5kdc[30854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:22.225885 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43149
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:22.343415 30868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:22.343691 30868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:22.343735 30868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:22.347841 30868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:22.347913 30868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:22.347939 30868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:22.347956 30868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:22.347971 30868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:22.354526 30868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:22.355654 30868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:22.356756 30868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:22.365868 30874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:22.365869 30877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:22.365868 30875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:22.366228 30868 server_base.cc:1034] running on GCE node
I20260809 04:49:22.366791 30868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:22.367950 30868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:22.369161 30868 hybrid_clock.cc:648] HybridClock initialized: now 1786250962369148 us; error 28 us; skew 500 ppm
Aug 09 04:49:22 dist-test-slave-1zqn krb5kdc[30854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250962, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:22.372164 30868 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:22.373159 30868 webserver.cc:460] Webserver started at http://127.0.0.1:38165/ using document root <none> and password file <none>
I20260809 04:49:22.373739 30868 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:22.373842 30868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:22.374086 30868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:22.376305 30868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "a83dcfe9b48e4579bc05bb2815495a17"
format_stamp: "Formatted at 2026-08-09 04:49:22 on dist-test-slave-1zqn"
server_key: "5fcd615887ce954efd5707ce13753f00"
server_key_iv: "614648ff9896b344d003f136dcfcf47e"
server_key_version: "encryptionkey@0"
I20260809 04:49:22.376880 30868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "a83dcfe9b48e4579bc05bb2815495a17"
format_stamp: "Formatted at 2026-08-09 04:49:22 on dist-test-slave-1zqn"
server_key: "5fcd615887ce954efd5707ce13753f00"
server_key_iv: "614648ff9896b344d003f136dcfcf47e"
server_key_version: "encryptionkey@0"
I20260809 04:49:22.380863 30868 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:49:22.383422 30885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:22.384760 30868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 04:49:22.384913 30868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "a83dcfe9b48e4579bc05bb2815495a17"
format_stamp: "Formatted at 2026-08-09 04:49:22 on dist-test-slave-1zqn"
server_key: "5fcd615887ce954efd5707ce13753f00"
server_key_iv: "614648ff9896b344d003f136dcfcf47e"
server_key_version: "encryptionkey@0"
I20260809 04:49:22.385042 30868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:22.446489 30868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:22.449764 30868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:22.449980 30868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:22.457442 30868 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43149
I20260809 04:49:22.457470 30939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43149 every 8 connection(s)
I20260809 04:49:22.458402 30868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:22.461192 30940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:22.464493 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 30868
I20260809 04:49:22.464625 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:22.464906 7852 external_mini_cluster.cc:1428] Setting key 75e74b72ade4bf64d77d2de4395f152a
I20260809 04:49:22.467224 30940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17: Bootstrap starting.
I20260809 04:49:22.469524 30940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:22.470278 30940 log.cc:824] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:22.472242 30940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17: No bootstrap required, opened a new log
Aug 09 04:49:22 dist-test-slave-1zqn krb5kdc[30854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:49:22.476667 30940 raft_consensus.cc:348] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a83dcfe9b48e4579bc05bb2815495a17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } }
I20260809 04:49:22.477001 30940 raft_consensus.cc:374] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:22.477069 30940 raft_consensus.cc:729] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a83dcfe9b48e4579bc05bb2815495a17, State: Initialized, Role: FOLLOWER
I20260809 04:49:22.477582 30940 consensus_queue.cc:260] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a83dcfe9b48e4579bc05bb2815495a17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } }
I20260809 04:49:22.477774 30940 raft_consensus.cc:388] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:22.477850 30940 raft_consensus.cc:482] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:22.477952 30940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:22.479130 30940 raft_consensus.cc:504] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a83dcfe9b48e4579bc05bb2815495a17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } }
I20260809 04:49:22.479530 30940 leader_election.cc:302] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a83dcfe9b48e4579bc05bb2815495a17; no voters:
I20260809 04:49:22.479884 30940 leader_election.cc:288] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:22.480007 30945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:22.480165 30943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.466378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58709 (local address 127.0.0.1:43149)
0809 04:49:22.466828 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.466844 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.466879 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:49:22.467663 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.467683 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.468031 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.468464 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.469268 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.470345 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.471231 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.471705 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.471882 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.474768 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.474773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.474787 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.474814 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.476700 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.477248 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.477254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.477260 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.477398 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.477690 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.477693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.477695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.478261 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.478454 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.478720 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.479062 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":143,"threads_started":1}
I20260809 04:49:22.480370 30945 raft_consensus.cc:686] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [term 1 LEADER]: Becoming Leader. State: Replica: a83dcfe9b48e4579bc05bb2815495a17, State: Running, Role: LEADER
I20260809 04:49:22.480764 30945 consensus_queue.cc:237] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a83dcfe9b48e4579bc05bb2815495a17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } }
I20260809 04:49:22.481232 30940 sys_catalog.cc:564] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:22.483655 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:22.484393 30946 sys_catalog.cc:455] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a83dcfe9b48e4579bc05bb2815495a17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a83dcfe9b48e4579bc05bb2815495a17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } } }
I20260809 04:49:22.484345 30947 sys_catalog.cc:455] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a83dcfe9b48e4579bc05bb2815495a17. Latest consensus state: current_term: 1 leader_uuid: "a83dcfe9b48e4579bc05bb2815495a17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a83dcfe9b48e4579bc05bb2815495a17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43149 } } }
I20260809 04:49:22.484546 30946 sys_catalog.cc:458] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:22.484606 30947 sys_catalog.cc:458] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17 [sys.catalog]: This master's current role is: LEADER
W20260809 04:49:22.486853 30960 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:22.486953 30960 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:22.487053 30962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:22.487738 30962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:22.494573 30962 catalog_manager.cc:1347] Generated new cluster ID: 17bb83fe90a44d7eba60f5d6fbb4b6fb
I20260809 04:49:22.494637 30962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:22.496328 30943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.486103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51559 (local address 127.0.0.1:43149)
0809 04:49:22.486521 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.486525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.486538 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:49:22.486574 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.486576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.486619 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.486700 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.487560 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.488441 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.489600 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.490029 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.490137 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.491896 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.491900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.491903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.491930 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.494280 (+ 2350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.494824 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.494828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.494830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.494921 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.495248 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.495252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.495254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.495466 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.495585 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.495995 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.496160 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343}
I20260809 04:49:22.517129 30962 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:22.518473 30962 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:22.547709 30962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a83dcfe9b48e4579bc05bb2815495a17: Generated new TSK 0
I20260809 04:49:22.548736 30962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:22.560333 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 30868
2026-08-09T04:49:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1615 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:22 dist-test-slave-1zqn krb5kdc[30973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:22 dist-test-slave-1zqn krb5kdc[30973](info): set up 2 sockets
Aug 09 04:49:22 dist-test-slave-1zqn krb5kdc[30973](info): commencing operation
krb5kdc: starting...
W20260809 04:49:23.691516 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:23 dist-test-slave-1zqn krb5kdc[30973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:23.827755 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:23.944679 30989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:23.944947 30989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:23.944989 30989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:23.948967 30989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:23.949031 30989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:23.949050 30989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:23.949067 30989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:23.949133 30989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:23.954535 30989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:23.955653 30989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:23.956743 30989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:23.965586 30994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:23.965610 30995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:23.965600 30997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:23.965821 30989 server_base.cc:1034] running on GCE node
I20260809 04:49:23.966454 30989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:23.967619 30989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:23.968819 30989 hybrid_clock.cc:648] HybridClock initialized: now 1786250963968788 us; error 49 us; skew 500 ppm
Aug 09 04:49:23 dist-test-slave-1zqn krb5kdc[30973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250963, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:23.976413 30989 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:23.977502 30989 webserver.cc:460] Webserver started at http://127.0.0.1:45449/ using document root <none> and password file <none>
I20260809 04:49:23.978152 30989 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:23.978261 30989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:23.978574 30989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:23.981171 30989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "aefcc992812a40e1a7e41683d9a88113"
format_stamp: "Formatted at 2026-08-09 04:49:23 on dist-test-slave-1zqn"
server_key: "1e4ff2bc584ca20bef1707209f83f2fe"
server_key_iv: "d804eeb3d078e5aad3608d0110ae39fe"
server_key_version: "encryptionkey@0"
I20260809 04:49:23.981777 30989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "aefcc992812a40e1a7e41683d9a88113"
format_stamp: "Formatted at 2026-08-09 04:49:23 on dist-test-slave-1zqn"
server_key: "1e4ff2bc584ca20bef1707209f83f2fe"
server_key_iv: "d804eeb3d078e5aad3608d0110ae39fe"
server_key_version: "encryptionkey@0"
I20260809 04:49:23.986019 30989 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 04:49:23.988756 31004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:23.990151 30989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:23.990320 30989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "aefcc992812a40e1a7e41683d9a88113"
format_stamp: "Formatted at 2026-08-09 04:49:23 on dist-test-slave-1zqn"
server_key: "1e4ff2bc584ca20bef1707209f83f2fe"
server_key_iv: "d804eeb3d078e5aad3608d0110ae39fe"
server_key_version: "encryptionkey@0"
I20260809 04:49:23.990460 30989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:24.026326 30989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:24.030197 30989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:24.030436 30989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:24.038709 30989 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39461
I20260809 04:49:24.038739 31065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39461 every 8 connection(s)
I20260809 04:49:24.039789 30989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:24.042714 31066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:24.046068 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 30989
I20260809 04:49:24.046259 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:24.046598 7852 external_mini_cluster.cc:1428] Setting key 3465d89672668821c53d2d0ab5a9d8d4
I20260809 04:49:24.048841 31066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113: Bootstrap starting.
I20260809 04:49:24.051421 31066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:24.052407 31066 log.cc:824] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:24.054728 31066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113: No bootstrap required, opened a new log
I20260809 04:49:24.056422 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:24.048166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42225 (local address 127.0.0.1:39461)
0809 04:49:24.048607 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:24.048623 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:24.048659 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:49:24.049365 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:24.049388 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:24.049750 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:24.050028 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:24.050372 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:24.051526 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:24.052764 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:24.053509 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:24.053646 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:24.054508 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:24.054514 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:24.054527 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:24.054985 (+ 458us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:24.055043 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:24.055066 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:24.055148 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:24.055366 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 04:49:24.055634 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":135,"threads_started":1}
I20260809 04:49:24.059041 31066 raft_consensus.cc:348] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefcc992812a40e1a7e41683d9a88113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39461 } }
I20260809 04:49:24.059345 31066 raft_consensus.cc:374] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:24.059412 31066 raft_consensus.cc:729] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aefcc992812a40e1a7e41683d9a88113, State: Initialized, Role: FOLLOWER
I20260809 04:49:24.059656 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:24.059909 31066 consensus_queue.cc:260] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefcc992812a40e1a7e41683d9a88113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39461 } }
I20260809 04:49:24.060041 31066 raft_consensus.cc:388] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:24.060148 31066 raft_consensus.cc:482] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:24.060253 31066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:24.061254 31066 raft_consensus.cc:504] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefcc992812a40e1a7e41683d9a88113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39461 } }
I20260809 04:49:24.061581 31066 leader_election.cc:302] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aefcc992812a40e1a7e41683d9a88113; no voters:
I20260809 04:49:24.061908 31066 leader_election.cc:288] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:24.062199 31075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:24.062384 31075 raft_consensus.cc:686] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [term 1 LEADER]: Becoming Leader. State: Replica: aefcc992812a40e1a7e41683d9a88113, State: Running, Role: LEADER
I20260809 04:49:24.062739 31075 consensus_queue.cc:237] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefcc992812a40e1a7e41683d9a88113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39461 } }
I20260809 04:49:24.063236 31066 sys_catalog.cc:564] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:24.063938 31077 sys_catalog.cc:455] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aefcc992812a40e1a7e41683d9a88113. Latest consensus state: current_term: 1 leader_uuid: "aefcc992812a40e1a7e41683d9a88113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefcc992812a40e1a7e41683d9a88113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39461 } } }
I20260809 04:49:24.064128 31077 sys_catalog.cc:458] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:24.064494 31078 sys_catalog.cc:455] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aefcc992812a40e1a7e41683d9a88113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aefcc992812a40e1a7e41683d9a88113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39461 } } }
I20260809 04:49:24.064599 31078 sys_catalog.cc:458] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:24.067016 31069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:24.061982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56911 (local address 127.0.0.1:39461)
0809 04:49:24.062095 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:24.062101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:24.062351 (+ 250us) server_negotiation.cc:407] Connection header received
0809 04:49:24.062521 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:24.062524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:24.062565 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:24.062613 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:24.063017 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:24.063821 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:24.065007 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:24.065331 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:24.065426 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:24.066520 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:24.066524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:24.066527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:24.066540 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:24.066566 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:24.066579 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:24.066653 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:24.066810 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 04:49:24.066869 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 04:49:24.068372 31093 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:24.068441 31093 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:24.068537 31084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:24.069229 31084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:24.074636 31084 catalog_manager.cc:1347] Generated new cluster ID: bac7cbeab3794bc6b5269c461c0f4dd8
I20260809 04:49:24.074710 31084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:24.088191 31084 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:24.089186 31084 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:24.094420 31084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aefcc992812a40e1a7e41683d9a88113: Generated new TSK 0
I20260809 04:49:24.095207 31084 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:24.103785 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 30989
2026-08-09T04:49:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1543 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:24 dist-test-slave-1zqn krb5kdc[31100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:24 dist-test-slave-1zqn krb5kdc[31100](info): set up 2 sockets
Aug 09 04:49:24 dist-test-slave-1zqn krb5kdc[31100](info): commencing operation
krb5kdc: starting...
W20260809 04:49:25.224259 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:25 dist-test-slave-1zqn krb5kdc[31100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:25.359194 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38365
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:25.474004 31115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:25.474364 31115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:25.474447 31115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:25.478596 31115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:25.478716 31115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:25.478758 31115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:25.478796 31115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:25.478832 31115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:25.484385 31115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:25.485581 31115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:25.486713 31115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:25.495550 31122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:25.495622 31115 server_base.cc:1034] running on GCE node
W20260809 04:49:25.495599 31121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:25.495572 31124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:25.496438 31115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:25.497611 31115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:25.498816 31115 hybrid_clock.cc:648] HybridClock initialized: now 1786250965498799 us; error 26 us; skew 500 ppm
Aug 09 04:49:25 dist-test-slave-1zqn krb5kdc[31100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250965, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:25.501689 31115 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:25.502642 31115 webserver.cc:460] Webserver started at http://127.0.0.1:35587/ using document root <none> and password file <none>
I20260809 04:49:25.503252 31115 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:25.503346 31115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:25.503589 31115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:25.505700 31115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "779adb43240941038da0a7226028cdf8"
format_stamp: "Formatted at 2026-08-09 04:49:25 on dist-test-slave-1zqn"
server_key: "637d1a0253aca927d9cfe6ef67950b6f"
server_key_iv: "6ef725855a2807ef24467adfab14626b"
server_key_version: "encryptionkey@0"
I20260809 04:49:25.506289 31115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "779adb43240941038da0a7226028cdf8"
format_stamp: "Formatted at 2026-08-09 04:49:25 on dist-test-slave-1zqn"
server_key: "637d1a0253aca927d9cfe6ef67950b6f"
server_key_iv: "6ef725855a2807ef24467adfab14626b"
server_key_version: "encryptionkey@0"
I20260809 04:49:25.510100 31115 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:49:25.512565 31133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:25.513867 31115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:25.514014 31115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "779adb43240941038da0a7226028cdf8"
format_stamp: "Formatted at 2026-08-09 04:49:25 on dist-test-slave-1zqn"
server_key: "637d1a0253aca927d9cfe6ef67950b6f"
server_key_iv: "6ef725855a2807ef24467adfab14626b"
server_key_version: "encryptionkey@0"
I20260809 04:49:25.514149 31115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:25.519078 31115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:25.522001 31115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:25.522197 31115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:25.529353 31115 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38365
I20260809 04:49:25.529454 31191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38365 every 8 connection(s)
I20260809 04:49:25.530397 31115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:25.533370 31192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:25.536702 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 31115
I20260809 04:49:25.536866 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:25.537163 7852 external_mini_cluster.cc:1428] Setting key 495730287986830df3e5ccc54dbf2145
I20260809 04:49:25.539544 31192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8: Bootstrap starting.
I20260809 04:49:25.542019 31192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:25.542670 31195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:25.538653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49667 (local address 127.0.0.1:38365)
0809 04:49:25.539159 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:25.539177 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:25.539222 (+ 45us) server_negotiation.cc:407] Connection header received
0809 04:49:25.539920 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:25.539938 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:25.540295 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:25.540640 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:25.541432 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:25.541436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:25.541449 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:25.541459 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:25.541520 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:25.541540 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:25.541657 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:25.541760 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 04:49:25.541926 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":156,"threads_started":1}
I20260809 04:49:25.542858 31192 log.cc:824] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:25.544907 31192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8: No bootstrap required, opened a new log
I20260809 04:49:25.546016 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:25.549080 31192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779adb43240941038da0a7226028cdf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38365 } }
I20260809 04:49:25.549345 31192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:25.549415 31192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779adb43240941038da0a7226028cdf8, State: Initialized, Role: FOLLOWER
I20260809 04:49:25.549849 31192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779adb43240941038da0a7226028cdf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38365 } }
I20260809 04:49:25.550041 31192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:25.550077 31195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:25.548240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57219 (local address 127.0.0.1:38365)
0809 04:49:25.548364 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:25.548368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:25.548601 (+ 233us) server_negotiation.cc:407] Connection header received
0809 04:49:25.548712 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:25.548714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:25.548744 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:25.548817 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:25.549577 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:25.549579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:25.549581 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:25.549592 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:25.549618 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:25.549628 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:25.549735 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:25.549816 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 04:49:25.549855 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 04:49:25.550122 31192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:25.550216 31192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:25.551224 31192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779adb43240941038da0a7226028cdf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38365 } }
I20260809 04:49:25.551556 31192 leader_election.cc:302] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 779adb43240941038da0a7226028cdf8; no voters:
I20260809 04:49:25.551859 31192 leader_election.cc:288] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:25.552124 31202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:25.552382 31202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [term 1 LEADER]: Becoming Leader. State: Replica: 779adb43240941038da0a7226028cdf8, State: Running, Role: LEADER
I20260809 04:49:25.552809 31202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779adb43240941038da0a7226028cdf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38365 } }
I20260809 04:49:25.553371 31192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:25.554718 31203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "779adb43240941038da0a7226028cdf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779adb43240941038da0a7226028cdf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38365 } } }
I20260809 04:49:25.554852 31203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:25.555210 31204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 779adb43240941038da0a7226028cdf8. Latest consensus state: current_term: 1 leader_uuid: "779adb43240941038da0a7226028cdf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779adb43240941038da0a7226028cdf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38365 } } }
I20260809 04:49:25.555289 31212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:25.555370 31204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:25.558182 31212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:25.563476 31212 catalog_manager.cc:1347] Generated new cluster ID: 9e9f411a60634325a145f6692ffceb3e
I20260809 04:49:25.563560 31212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:25.570051 31212 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:25.570365 31212 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:25.576737 31212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 779adb43240941038da0a7226028cdf8: Generated new TSK 0
I20260809 04:49:25.577546 31212 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:25.591039 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 31115
2026-08-09T04:49:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1484 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:25 dist-test-slave-1zqn krb5kdc[31223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:25 dist-test-slave-1zqn krb5kdc[31223](info): set up 2 sockets
Aug 09 04:49:25 dist-test-slave-1zqn krb5kdc[31223](info): commencing operation
krb5kdc: starting...
W20260809 04:49:26.694383 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:26 dist-test-slave-1zqn krb5kdc[31223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:26.831552 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34077
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:26.945955 31236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:26.946322 31236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:26.946408 31236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:26.950649 31236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:26.950757 31236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:26.950798 31236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:26.950850 31236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:26.950886 31236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:26.956432 31236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:26.957612 31236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:26.958771 31236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:26.967635 31241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:26.967643 31243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:26.967656 31236 server_base.cc:1034] running on GCE node
W20260809 04:49:26.967784 31245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:26.968542 31236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:26.969607 31236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:26.970772 31236 hybrid_clock.cc:648] HybridClock initialized: now 1786250966970730 us; error 56 us; skew 500 ppm
Aug 09 04:49:26 dist-test-slave-1zqn krb5kdc[31223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250966, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:26.973723 31236 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:26.974572 31236 webserver.cc:460] Webserver started at http://127.0.0.1:38859/ using document root <none> and password file <none>
I20260809 04:49:26.975101 31236 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:26.975162 31236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:26.975332 31236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:26.977469 31236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "cf2021506e224a2ba73b7e9b302692e7"
format_stamp: "Formatted at 2026-08-09 04:49:26 on dist-test-slave-1zqn"
server_key: "5c9caf67ef5043e5821fe8cc2bfdc9a8"
server_key_iv: "3981a0f690df56b68b49cf42dbaab490"
server_key_version: "encryptionkey@0"
I20260809 04:49:26.978024 31236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "cf2021506e224a2ba73b7e9b302692e7"
format_stamp: "Formatted at 2026-08-09 04:49:26 on dist-test-slave-1zqn"
server_key: "5c9caf67ef5043e5821fe8cc2bfdc9a8"
server_key_iv: "3981a0f690df56b68b49cf42dbaab490"
server_key_version: "encryptionkey@0"
I20260809 04:49:26.982503 31236 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 04:49:26.985076 31254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:26.986481 31236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:26.986645 31236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "cf2021506e224a2ba73b7e9b302692e7"
format_stamp: "Formatted at 2026-08-09 04:49:26 on dist-test-slave-1zqn"
server_key: "5c9caf67ef5043e5821fe8cc2bfdc9a8"
server_key_iv: "3981a0f690df56b68b49cf42dbaab490"
server_key_version: "encryptionkey@0"
I20260809 04:49:26.986779 31236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:27.005937 31236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:27.009372 31236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:27.009603 31236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:27.017016 31236 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34077
I20260809 04:49:27.017115 31314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34077 every 8 connection(s)
I20260809 04:49:27.018025 31236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:27.019214 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 31236
I20260809 04:49:27.019399 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:27.019733 7852 external_mini_cluster.cc:1428] Setting key 76b6854dc57a69cfa835c2e601d7e382
I20260809 04:49:27.021364 31315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:27.028333 31315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7: Bootstrap starting.
Aug 09 04:49:27 dist-test-slave-1zqn krb5kdc[31223](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:49:27.031069 31315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:27.032115 31315 log.cc:824] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:27.034584 31315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7: No bootstrap required, opened a new log
I20260809 04:49:27.036007 31319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.021201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:48375 (local address 127.0.0.1:34077)
0809 04:49:27.021679 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.021696 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.021736 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:49:27.022467 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.022485 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.022867 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.023197 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.024217 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.025324 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.026588 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.027210 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.027359 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.030601 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.030608 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.030622 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.030652 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.032787 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.033333 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.033340 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.033346 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.033482 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.033804 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.033807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.033809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.034319 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.034518 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.034830 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.035082 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":161,"threads_started":1}
I20260809 04:49:27.039081 31315 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2021506e224a2ba73b7e9b302692e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34077 } }
I20260809 04:49:27.039395 31315 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:27.039412 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:27.039490 31315 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf2021506e224a2ba73b7e9b302692e7, State: Initialized, Role: FOLLOWER
I20260809 04:49:27.039966 31315 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2021506e224a2ba73b7e9b302692e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34077 } }
I20260809 04:49:27.040144 31315 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:27.040217 31315 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:27.040318 31315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:27.041396 31315 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2021506e224a2ba73b7e9b302692e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34077 } }
I20260809 04:49:27.041779 31315 leader_election.cc:302] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf2021506e224a2ba73b7e9b302692e7; no voters:
I20260809 04:49:27.042183 31315 leader_election.cc:288] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:27.042272 31326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:27.042513 31326 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [term 1 LEADER]: Becoming Leader. State: Replica: cf2021506e224a2ba73b7e9b302692e7, State: Running, Role: LEADER
I20260809 04:49:27.042860 31326 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2021506e224a2ba73b7e9b302692e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34077 } }
I20260809 04:49:27.043584 31315 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:27.044158 31326 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf2021506e224a2ba73b7e9b302692e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2021506e224a2ba73b7e9b302692e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34077 } } }
I20260809 04:49:27.044334 31326 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:27.044562 31327 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf2021506e224a2ba73b7e9b302692e7. Latest consensus state: current_term: 1 leader_uuid: "cf2021506e224a2ba73b7e9b302692e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf2021506e224a2ba73b7e9b302692e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34077 } } }
I20260809 04:49:27.044668 31327 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:27.044811 31332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:27.048264 31332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:27.051488 31319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.041729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:51903 (local address 127.0.0.1:34077)
0809 04:49:27.041861 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.041884 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.042117 (+ 233us) server_negotiation.cc:407] Connection header received
0809 04:49:27.042257 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.042260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.042309 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.042401 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.043242 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.043913 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.045159 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.045518 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.045599 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.047858 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.047862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.047864 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.047897 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.049709 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.050200 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.050204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.050206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.050292 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.050630 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.050633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.050635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.050814 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.050977 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.051246 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.051339 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:49:27.054935 31332 catalog_manager.cc:1347] Generated new cluster ID: 000e3e8d7dd5436cb5591e15298c6ef1
I20260809 04:49:27.055078 31332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:27.083873 31332 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:27.084971 31332 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:27.095445 31332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf2021506e224a2ba73b7e9b302692e7: Generated new TSK 0
I20260809 04:49:27.096276 31332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:27.102198 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 31236
W20260809 04:49:27.111881 31221 connection.cc:535] client connection to 127.0.0.1:34077 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34077): Connection reset by peer (error 104)
2026-08-09T04:49:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:27 dist-test-slave-1zqn krb5kdc[31355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:27 dist-test-slave-1zqn krb5kdc[31355](info): set up 2 sockets
Aug 09 04:49:27 dist-test-slave-1zqn krb5kdc[31355](info): commencing operation
krb5kdc: starting...
W20260809 04:49:28.203747 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:28 dist-test-slave-1zqn krb5kdc[31355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:28.347316 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34829
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:28.464272 31369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:28.464550 31369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:28.464594 31369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:28.468762 31369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:28.468830 31369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:28.468852 31369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:28.468869 31369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:28.468921 31369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:28.474373 31369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:28.475630 31369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:28.476768 31369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:28.485656 31375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:28.485832 31369 server_base.cc:1034] running on GCE node
W20260809 04:49:28.485906 31374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:28.485849 31377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:28.486691 31369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:28.487818 31369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:28.489037 31369 hybrid_clock.cc:648] HybridClock initialized: now 1786250968489009 us; error 46 us; skew 500 ppm
Aug 09 04:49:28 dist-test-slave-1zqn krb5kdc[31355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:28.491874 31369 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:28.492828 31369 webserver.cc:460] Webserver started at http://127.0.0.1:42363/ using document root <none> and password file <none>
I20260809 04:49:28.493360 31369 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:28.493419 31369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:28.493589 31369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:28.495697 31369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "ed90725c1bcc476aa9b65937324da8f9"
format_stamp: "Formatted at 2026-08-09 04:49:28 on dist-test-slave-1zqn"
server_key: "f0bdb6ec5bd97b774856c7872efacf89"
server_key_iv: "9aad251843816dc76b85580d479167d6"
server_key_version: "encryptionkey@0"
I20260809 04:49:28.496369 31369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "ed90725c1bcc476aa9b65937324da8f9"
format_stamp: "Formatted at 2026-08-09 04:49:28 on dist-test-slave-1zqn"
server_key: "f0bdb6ec5bd97b774856c7872efacf89"
server_key_iv: "9aad251843816dc76b85580d479167d6"
server_key_version: "encryptionkey@0"
I20260809 04:49:28.500250 31369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:49:28.502803 31384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:28.504292 31369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:28.504441 31369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "ed90725c1bcc476aa9b65937324da8f9"
format_stamp: "Formatted at 2026-08-09 04:49:28 on dist-test-slave-1zqn"
server_key: "f0bdb6ec5bd97b774856c7872efacf89"
server_key_iv: "9aad251843816dc76b85580d479167d6"
server_key_version: "encryptionkey@0"
I20260809 04:49:28.504567 31369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:28.541435 31369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:28.544931 31369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:28.545104 31369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:28.552585 31369 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34829
I20260809 04:49:28.552613 31442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34829 every 8 connection(s)
I20260809 04:49:28.553560 31369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:28.555824 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 31369
I20260809 04:49:28.556030 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:28.556376 7852 external_mini_cluster.cc:1428] Setting key da979cc671f3515d627cedad04d0e5a3
I20260809 04:49:28.556658 31443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:28.563323 31443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9: Bootstrap starting.
I20260809 04:49:28.565841 31443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:28.566648 31443 log.cc:824] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:28.567059 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:28.557914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:43643 (local address 127.0.0.1:34829)
0809 04:49:28.558406 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.558421 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.558464 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:49:28.559487 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.559508 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.559898 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.560405 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:28.560865 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.562167 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.563178 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.563775 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.563980 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:28.565053 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:28.565058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:28.565072 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:28.565494 (+ 422us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:28.565553 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:28.565577 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:28.565737 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:28.565979 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 04:49:28.566144 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":155,"threads_started":1}
I20260809 04:49:28.568907 31443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9: No bootstrap required, opened a new log
I20260809 04:49:28.570410 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:28.573006 31443 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed90725c1bcc476aa9b65937324da8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } }
I20260809 04:49:28.573331 31443 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:28.573400 31443 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed90725c1bcc476aa9b65937324da8f9, State: Initialized, Role: FOLLOWER
I20260809 04:49:28.573860 31443 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed90725c1bcc476aa9b65937324da8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } }
I20260809 04:49:28.573992 31443 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:28.574087 31443 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:28.574208 31443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:28.575387 31443 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed90725c1bcc476aa9b65937324da8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } }
I20260809 04:49:28.575774 31443 leader_election.cc:302] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed90725c1bcc476aa9b65937324da8f9; no voters:
I20260809 04:49:28.576130 31443 leader_election.cc:288] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:28.576203 31456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:28.576426 31456 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [term 1 LEADER]: Becoming Leader. State: Replica: ed90725c1bcc476aa9b65937324da8f9, State: Running, Role: LEADER
I20260809 04:49:28.577121 31456 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed90725c1bcc476aa9b65937324da8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } }
I20260809 04:49:28.577507 31443 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:28.577775 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:28.572681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:36191 (local address 127.0.0.1:34829)
0809 04:49:28.572824 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.572828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.573043 (+ 215us) server_negotiation.cc:407] Connection header received
0809 04:49:28.573179 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.573181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.573222 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.573336 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:28.573614 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.574317 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.575510 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.575835 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.575990 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:28.577170 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:28.577174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:28.577176 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:28.577190 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:28.577246 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:28.577262 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:28.577351 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:28.577570 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 04:49:28.577609 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:49:28.578835 31458 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed90725c1bcc476aa9b65937324da8f9. Latest consensus state: current_term: 1 leader_uuid: "ed90725c1bcc476aa9b65937324da8f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed90725c1bcc476aa9b65937324da8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } } }
I20260809 04:49:28.578969 31458 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:28.579183 31457 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed90725c1bcc476aa9b65937324da8f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed90725c1bcc476aa9b65937324da8f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34829 } } }
I20260809 04:49:28.579450 31457 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:28.579644 31467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:28.583107 31467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:28.588945 31467 catalog_manager.cc:1347] Generated new cluster ID: 0302e4052efc4c27aaa95327fc5200f8
I20260809 04:49:28.589025 31467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:28.595754 31467 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:28.596837 31467 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:28.624739 31467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed90725c1bcc476aa9b65937324da8f9: Generated new TSK 0
I20260809 04:49:28.625877 31467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:28.635950 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 31369
W20260809 04:49:28.643764 31353 connection.cc:535] client connection to 127.0.0.1:34829 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34829): Connection reset by peer (error 104)
2026-08-09T04:49:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:28 dist-test-slave-1zqn krb5kdc[31478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:28 dist-test-slave-1zqn krb5kdc[31478](info): set up 2 sockets
Aug 09 04:49:28 dist-test-slave-1zqn krb5kdc[31478](info): commencing operation
krb5kdc: starting...
W20260809 04:49:29.752894 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:29 dist-test-slave-1zqn krb5kdc[31478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:29.892350 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37935
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:30.007683 31493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:30.008052 31493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:30.008164 31493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:30.012418 31493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:30.012514 31493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:30.012552 31493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:30.012605 31493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:30.012655 31493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:30.017992 31493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:30.019130 31493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:30.020254 31493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:30.029142 31500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:30.029125 31498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:30.029227 31502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:30.029412 31493 server_base.cc:1034] running on GCE node
I20260809 04:49:30.030089 31493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:30.031345 31493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:30.032546 31493 hybrid_clock.cc:648] HybridClock initialized: now 1786250970032514 us; error 47 us; skew 500 ppm
Aug 09 04:49:30 dist-test-slave-1zqn krb5kdc[31478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250970, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:30.042265 31493 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:30.043365 31493 webserver.cc:460] Webserver started at http://127.0.0.1:36405/ using document root <none> and password file <none>
I20260809 04:49:30.044034 31493 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:30.044200 31493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:30.044457 31493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:30.046643 31493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "3c6b75659e244b9db1ced2bf2267fd76"
format_stamp: "Formatted at 2026-08-09 04:49:30 on dist-test-slave-1zqn"
server_key: "9bb8cccc823ffe85465a604f1d236f34"
server_key_iv: "25db7a8f0daa2a15c45e097d5187d4b4"
server_key_version: "encryptionkey@0"
I20260809 04:49:30.047245 31493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "3c6b75659e244b9db1ced2bf2267fd76"
format_stamp: "Formatted at 2026-08-09 04:49:30 on dist-test-slave-1zqn"
server_key: "9bb8cccc823ffe85465a604f1d236f34"
server_key_iv: "25db7a8f0daa2a15c45e097d5187d4b4"
server_key_version: "encryptionkey@0"
I20260809 04:49:30.051234 31493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 04:49:30.053928 31509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:30.055235 31493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:49:30.055392 31493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "3c6b75659e244b9db1ced2bf2267fd76"
format_stamp: "Formatted at 2026-08-09 04:49:30 on dist-test-slave-1zqn"
server_key: "9bb8cccc823ffe85465a604f1d236f34"
server_key_iv: "25db7a8f0daa2a15c45e097d5187d4b4"
server_key_version: "encryptionkey@0"
I20260809 04:49:30.055519 31493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:30.060362 31493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:30.063426 31493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:30.063637 31493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:30.071472 31493 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37935
I20260809 04:49:30.071488 31567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37935 every 8 connection(s)
I20260809 04:49:30.072505 31493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:30.075394 31568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:30.080360 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 31493
I20260809 04:49:30.080585 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:30.080870 7852 external_mini_cluster.cc:1428] Setting key b192e6e6a815d4af6c704a653709451e
I20260809 04:49:30.081907 31568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76: Bootstrap starting.
I20260809 04:49:30.084849 31568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:30.085903 31568 log.cc:824] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:30.087255 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:30.082534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:39531 (local address 127.0.0.1:37935)
0809 04:49:30.082990 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:30.083008 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:30.083065 (+ 57us) server_negotiation.cc:407] Connection header received
0809 04:49:30.083950 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:30.083975 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:30.084378 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:30.084755 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:30.085818 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:30.085824 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:30.085839 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:30.085849 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:30.085910 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:30.085942 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:30.086066 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:30.086222 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 04:49:30.086398 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":117,"threads_started":1}
I20260809 04:49:30.088215 31568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76: No bootstrap required, opened a new log
I20260809 04:49:30.090422 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:30.092792 31568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6b75659e244b9db1ced2bf2267fd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } }
I20260809 04:49:30.093091 31568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:30.093199 31568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c6b75659e244b9db1ced2bf2267fd76, State: Initialized, Role: FOLLOWER
I20260809 04:49:30.093706 31568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6b75659e244b9db1ced2bf2267fd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } }
I20260809 04:49:30.093889 31568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:30.093971 31568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:30.094116 31568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:30.094622 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:30.092697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:45783 (local address 127.0.0.1:37935)
0809 04:49:30.092824 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:30.092829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:30.093014 (+ 185us) server_negotiation.cc:407] Connection header received
0809 04:49:30.093173 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:30.093176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:30.093221 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:30.093313 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:30.094207 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:30.094211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:30.094213 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:30.094226 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:30.094258 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:30.094273 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:30.094378 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:30.094462 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 04:49:30.094506 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:49:30.095410 31568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6b75659e244b9db1ced2bf2267fd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } }
I20260809 04:49:30.095796 31568 leader_election.cc:302] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c6b75659e244b9db1ced2bf2267fd76; no voters:
I20260809 04:49:30.096151 31568 leader_election.cc:288] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:30.096261 31579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:30.096493 31579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [term 1 LEADER]: Becoming Leader. State: Replica: 3c6b75659e244b9db1ced2bf2267fd76, State: Running, Role: LEADER
I20260809 04:49:30.096892 31579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6b75659e244b9db1ced2bf2267fd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } }
I20260809 04:49:30.097491 31568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:30.098623 31580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c6b75659e244b9db1ced2bf2267fd76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6b75659e244b9db1ced2bf2267fd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } } }
I20260809 04:49:30.098773 31580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:30.099171 31589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:30.099637 31581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c6b75659e244b9db1ced2bf2267fd76. Latest consensus state: current_term: 1 leader_uuid: "3c6b75659e244b9db1ced2bf2267fd76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6b75659e244b9db1ced2bf2267fd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37935 } } }
I20260809 04:49:30.099737 31581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:30.102413 31589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:30.108032 31589 catalog_manager.cc:1347] Generated new cluster ID: cfb9aaba149943f2b0583c033315f64e
I20260809 04:49:30.108175 31589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:30.137302 31589 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:30.137696 31589 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:30.151871 31589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c6b75659e244b9db1ced2bf2267fd76: Generated new TSK 0
I20260809 04:49:30.152776 31589 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:30.163440 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 31493
2026-08-09T04:49:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:30 dist-test-slave-1zqn krb5kdc[31603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:30 dist-test-slave-1zqn krb5kdc[31603](info): set up 2 sockets
Aug 09 04:49:30 dist-test-slave-1zqn krb5kdc[31603](info): commencing operation
krb5kdc: starting...
W20260809 04:49:31.266950 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:31 dist-test-slave-1zqn krb5kdc[31603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:31.402884 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32847
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:31.519109 31616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:31.519426 31616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:31.519510 31616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:31.523643 31616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:31.523748 31616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:31.523789 31616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:31.523849 31616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:31.523890 31616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:31.529520 31616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32847
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:31.530664 31616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:31.531733 31616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:31.540812 31622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:31.540858 31623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:31.540865 31626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:31.541172 31616 server_base.cc:1034] running on GCE node
I20260809 04:49:31.541754 31616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:31.542927 31616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:31.544133 31616 hybrid_clock.cc:648] HybridClock initialized: now 1786250971544120 us; error 28 us; skew 500 ppm
Aug 09 04:49:31 dist-test-slave-1zqn krb5kdc[31603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250971, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:31.546960 31616 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:31.547811 31616 webserver.cc:460] Webserver started at http://127.0.0.1:34495/ using document root <none> and password file <none>
I20260809 04:49:31.548425 31616 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:31.548491 31616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:31.548661 31616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:31.550800 31616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "aa3e10b3981e46bdb5c60f1aafe81f49"
format_stamp: "Formatted at 2026-08-09 04:49:31 on dist-test-slave-1zqn"
server_key: "18f733cd84c037f3399e78e6dd30f5f1"
server_key_iv: "05844df205d3126f8235336b01d1672d"
server_key_version: "encryptionkey@0"
I20260809 04:49:31.551319 31616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "aa3e10b3981e46bdb5c60f1aafe81f49"
format_stamp: "Formatted at 2026-08-09 04:49:31 on dist-test-slave-1zqn"
server_key: "18f733cd84c037f3399e78e6dd30f5f1"
server_key_iv: "05844df205d3126f8235336b01d1672d"
server_key_version: "encryptionkey@0"
I20260809 04:49:31.555102 31616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 04:49:31.557652 31633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:31.559037 31616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:49:31.559192 31616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "aa3e10b3981e46bdb5c60f1aafe81f49"
format_stamp: "Formatted at 2026-08-09 04:49:31 on dist-test-slave-1zqn"
server_key: "18f733cd84c037f3399e78e6dd30f5f1"
server_key_iv: "05844df205d3126f8235336b01d1672d"
server_key_version: "encryptionkey@0"
I20260809 04:49:31.559314 31616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:31.571138 31616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:31.574383 31616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:31.574592 31616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:31.582198 31616 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32847
I20260809 04:49:31.582247 31689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32847 every 8 connection(s)
I20260809 04:49:31.583211 31616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:31.586489 31690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:31.590636 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 31616
I20260809 04:49:31.590803 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:31.591138 7852 external_mini_cluster.cc:1428] Setting key 32dd19e7aeea1dd913b452ccf71adfdb
I20260809 04:49:31.593230 31690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49: Bootstrap starting.
I20260809 04:49:31.596031 31690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:31.596915 31690 log.cc:824] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:31.599331 31690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49: No bootstrap required, opened a new log
Aug 09 04:49:31 dist-test-slave-1zqn krb5kdc[31603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:49:31.603820 31690 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3e10b3981e46bdb5c60f1aafe81f49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } }
I20260809 04:49:31.604116 31690 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:31.604188 31690 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3e10b3981e46bdb5c60f1aafe81f49, State: Initialized, Role: FOLLOWER
I20260809 04:49:31.604739 31690 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3e10b3981e46bdb5c60f1aafe81f49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } }
I20260809 04:49:31.604875 31690 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:31.604979 31690 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:31.605106 31690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:31.606086 31690 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3e10b3981e46bdb5c60f1aafe81f49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } }
I20260809 04:49:31.606539 31690 leader_election.cc:302] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa3e10b3981e46bdb5c60f1aafe81f49; no voters:
I20260809 04:49:31.606982 31690 leader_election.cc:288] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:31.607112 31696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:31.607414 31696 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [term 1 LEADER]: Becoming Leader. State: Replica: aa3e10b3981e46bdb5c60f1aafe81f49, State: Running, Role: LEADER
I20260809 04:49:31.607810 31696 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3e10b3981e46bdb5c60f1aafe81f49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } }
I20260809 04:49:31.608475 31690 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:31.608683 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:31.592807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:50053 (local address 127.0.0.1:32847)
0809 04:49:31.593273 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:31.593291 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:31.593328 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:49:31.594085 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:31.594103 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:31.594502 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:31.594806 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:31.595854 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:31.597269 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:31.598502 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:31.599020 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:31.599148 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:31.602386 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:31.602391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:31.602405 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:31.602449 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:31.604606 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:31.605251 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:31.605260 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:31.605271 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:31.605390 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:31.605773 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:31.605782 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:31.605784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:31.606195 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:31.606475 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:31.607302 (+ 827us) server_negotiation.cc:299] Negotiation successful
0809 04:49:31.607577 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":152,"threads_started":1}
I20260809 04:49:31.610010 31698 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3e10b3981e46bdb5c60f1aafe81f49. Latest consensus state: current_term: 1 leader_uuid: "aa3e10b3981e46bdb5c60f1aafe81f49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3e10b3981e46bdb5c60f1aafe81f49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } } }
I20260809 04:49:31.610136 31697 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa3e10b3981e46bdb5c60f1aafe81f49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3e10b3981e46bdb5c60f1aafe81f49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32847 } } }
I20260809 04:49:31.610204 31698 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:31.610245 31697 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:31.610646 31705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:31.612224 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:31.614722 31705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:31.621629 31705 catalog_manager.cc:1347] Generated new cluster ID: 5758afd9e15641f7bb5aafe03dd535dc
I20260809 04:49:31.622013 31705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:31.625021 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:31.614742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:46127 (local address 127.0.0.1:32847)
0809 04:49:31.614861 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:31.614866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:31.615136 (+ 270us) server_negotiation.cc:407] Connection header received
0809 04:49:31.615254 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:31.615257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:31.615310 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:31.615409 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:31.616344 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:31.617217 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:31.618101 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:31.618434 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:31.618575 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:31.620736 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:31.620740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:31.620744 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:31.620778 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:31.622729 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:31.623366 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:31.623370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:31.623372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:31.623472 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:31.623864 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:31.623868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:31.623869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:31.624104 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:31.624260 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:31.624591 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 04:49:31.624695 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:49:31.653916 31705 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:31.654951 31705 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:31.674429 31705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa3e10b3981e46bdb5c60f1aafe81f49: Generated new TSK 0
I20260809 04:49:31.675274 31705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:31.683815 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 31616
W20260809 04:49:31.694528 31601 connection.cc:535] client connection to 127.0.0.1:32847 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32847): Connection reset by peer (error 104)
2026-08-09T04:49:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1522 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:31 dist-test-slave-1zqn krb5kdc[31722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:31 dist-test-slave-1zqn krb5kdc[31722](info): set up 2 sockets
Aug 09 04:49:31 dist-test-slave-1zqn krb5kdc[31722](info): commencing operation
krb5kdc: starting...
W20260809 04:49:32.883965 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:32 dist-test-slave-1zqn krb5kdc[31722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:33.023458 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36119
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:33.140908 31735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:33.141232 31735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:33.141306 31735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:33.145351 31735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:33.145447 31735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:33.145488 31735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:33.145545 31735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:33.145583 31735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:33.150949 31735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36119
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:33.152091 31735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:33.153158 31735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:33.162220 31747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:33.162194 31744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:33.162261 31735 server_base.cc:1034] running on GCE node
W20260809 04:49:33.162196 31743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:33.163041 31735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:33.164295 31735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:33.165491 31735 hybrid_clock.cc:648] HybridClock initialized: now 1786250973165479 us; error 26 us; skew 500 ppm
Aug 09 04:49:33 dist-test-slave-1zqn krb5kdc[31722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250973, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:33.168382 31735 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:33.169358 31735 webserver.cc:460] Webserver started at http://127.0.0.1:36857/ using document root <none> and password file <none>
I20260809 04:49:33.169931 31735 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:33.170066 31735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:33.170303 31735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:33.172463 31735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "e5acffe660d3460fb6cc6df123d8a973"
format_stamp: "Formatted at 2026-08-09 04:49:33 on dist-test-slave-1zqn"
server_key: "f383695d3c8d73780338f9ec2c90ab36"
server_key_iv: "2a69fa52e41760690f4e2d212ed3785a"
server_key_version: "encryptionkey@0"
I20260809 04:49:33.173141 31735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "e5acffe660d3460fb6cc6df123d8a973"
format_stamp: "Formatted at 2026-08-09 04:49:33 on dist-test-slave-1zqn"
server_key: "f383695d3c8d73780338f9ec2c90ab36"
server_key_iv: "2a69fa52e41760690f4e2d212ed3785a"
server_key_version: "encryptionkey@0"
I20260809 04:49:33.177047 31735 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:49:33.179574 31754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:33.180925 31735 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:33.181118 31735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "e5acffe660d3460fb6cc6df123d8a973"
format_stamp: "Formatted at 2026-08-09 04:49:33 on dist-test-slave-1zqn"
server_key: "f383695d3c8d73780338f9ec2c90ab36"
server_key_iv: "2a69fa52e41760690f4e2d212ed3785a"
server_key_version: "encryptionkey@0"
I20260809 04:49:33.181242 31735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:33.199308 31735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:33.202821 31735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:33.203064 31735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:33.211476 31735 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36119
I20260809 04:49:33.211520 31811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36119 every 8 connection(s)
I20260809 04:49:33.212553 31735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:33.215523 31812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:33.221364 31812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973: Bootstrap starting.
I20260809 04:49:33.221716 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 31735
I20260809 04:49:33.221861 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:33.222170 7852 external_mini_cluster.cc:1428] Setting key d9a9437716a759522912d3c606ba811c
I20260809 04:49:33.224211 31812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:33.224987 31812 log.cc:824] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:33.227284 31812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973: No bootstrap required, opened a new log
I20260809 04:49:33.231750 31812 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5acffe660d3460fb6cc6df123d8a973" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36119 } }
I20260809 04:49:33.232108 31812 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:33.232182 31812 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5acffe660d3460fb6cc6df123d8a973, State: Initialized, Role: FOLLOWER
I20260809 04:49:33.232633 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:33.223610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:55641 (local address 127.0.0.1:36119)
0809 04:49:33.224121 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:33.224138 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:33.224174 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:49:33.224875 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:33.224894 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:33.225299 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:33.225618 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:33.226034 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:33.227431 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:33.228518 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:33.229245 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:33.229413 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:33.230394 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:33.230400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:33.230418 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:33.230961 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:33.231033 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:33.231064 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:33.231207 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:33.231434 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 04:49:33.231587 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":140,"threads_started":1}
I20260809 04:49:33.232681 31812 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5acffe660d3460fb6cc6df123d8a973" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36119 } }
I20260809 04:49:33.232824 31812 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:33.232897 31812 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:33.233006 31812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:33.234138 31812 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5acffe660d3460fb6cc6df123d8a973" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36119 } }
I20260809 04:49:33.234511 31812 leader_election.cc:302] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5acffe660d3460fb6cc6df123d8a973; no voters:
I20260809 04:49:33.234890 31812 leader_election.cc:288] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:33.235008 31819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:33.235299 31819 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [term 1 LEADER]: Becoming Leader. State: Replica: e5acffe660d3460fb6cc6df123d8a973, State: Running, Role: LEADER
I20260809 04:49:33.235705 31819 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5acffe660d3460fb6cc6df123d8a973" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36119 } }
I20260809 04:49:33.236095 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:33.236331 31812 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:33.239568 31820 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5acffe660d3460fb6cc6df123d8a973" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5acffe660d3460fb6cc6df123d8a973" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36119 } } }
I20260809 04:49:33.239718 31820 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:33.239830 31821 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5acffe660d3460fb6cc6df123d8a973. Latest consensus state: current_term: 1 leader_uuid: "e5acffe660d3460fb6cc6df123d8a973" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5acffe660d3460fb6cc6df123d8a973" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36119 } } }
I20260809 04:49:33.239920 31821 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973 [sys.catalog]: This master's current role is: LEADER
W20260809 04:49:33.242044 31840 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:33.242173 31840 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:33.242416 31843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:33.243482 31843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:33.243755 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:33.238464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:47733 (local address 127.0.0.1:36119)
0809 04:49:33.238589 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:33.238593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:33.239344 (+ 751us) server_negotiation.cc:407] Connection header received
0809 04:49:33.239560 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:33.239564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:33.239614 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:33.239697 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:33.240102 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:33.240729 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:33.241722 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:33.241966 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:33.242081 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:33.243024 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:33.243028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:33.243031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:33.243047 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:33.243077 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:33.243090 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:33.243174 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:33.243594 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 04:49:33.243630 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:49:33.249056 31843 catalog_manager.cc:1347] Generated new cluster ID: e4c12df724084472ae46271dc585ed23
I20260809 04:49:33.249132 31843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:33.263461 31843 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:33.264492 31843 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:33.276183 31843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5acffe660d3460fb6cc6df123d8a973: Generated new TSK 0
I20260809 04:49:33.276943 31843 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:33.281579 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 31735
W20260809 04:49:33.290939 31720 connection.cc:535] client connection to 127.0.0.1:36119 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36119): Connection reset by peer (error 104)
2026-08-09T04:49:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1596 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:33 dist-test-slave-1zqn krb5kdc[31849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:33 dist-test-slave-1zqn krb5kdc[31849](info): set up 2 sockets
Aug 09 04:49:33 dist-test-slave-1zqn krb5kdc[31849](info): commencing operation
krb5kdc: starting...
W20260809 04:49:34.406421 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:34 dist-test-slave-1zqn krb5kdc[31849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:34.551251 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43121
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:34.666517 31862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:34.666836 31862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:34.666916 31862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:34.670995 31862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:34.671103 31862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:34.671140 31862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:34.671196 31862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:34.671233 31862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:34.676815 31862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43121
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:34.677932 31862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:34.679023 31862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:34.688028 31870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:34.688102 31862 server_base.cc:1034] running on GCE node
W20260809 04:49:34.687952 31867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:34.687952 31868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:34.688999 31862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:34.690146 31862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:34.691349 31862 hybrid_clock.cc:648] HybridClock initialized: now 1786250974691327 us; error 36 us; skew 500 ppm
Aug 09 04:49:34 dist-test-slave-1zqn krb5kdc[31849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250974, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:34.694265 31862 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:34.695188 31862 webserver.cc:460] Webserver started at http://127.0.0.1:46429/ using document root <none> and password file <none>
I20260809 04:49:34.695770 31862 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:34.695861 31862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:34.696161 31862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:34.698246 31862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "a8c60a0fd7584757ae7cfdf0d0a90009"
format_stamp: "Formatted at 2026-08-09 04:49:34 on dist-test-slave-1zqn"
server_key: "39e80dc86134430bc1ccddc1e91dbeae"
server_key_iv: "11a1ba8d92f08ca9dccdd336a09d685c"
server_key_version: "encryptionkey@0"
I20260809 04:49:34.698848 31862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "a8c60a0fd7584757ae7cfdf0d0a90009"
format_stamp: "Formatted at 2026-08-09 04:49:34 on dist-test-slave-1zqn"
server_key: "39e80dc86134430bc1ccddc1e91dbeae"
server_key_iv: "11a1ba8d92f08ca9dccdd336a09d685c"
server_key_version: "encryptionkey@0"
I20260809 04:49:34.702780 31862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:49:34.705578 31879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:34.707254 31862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:49:34.707429 31862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "a8c60a0fd7584757ae7cfdf0d0a90009"
format_stamp: "Formatted at 2026-08-09 04:49:34 on dist-test-slave-1zqn"
server_key: "39e80dc86134430bc1ccddc1e91dbeae"
server_key_iv: "11a1ba8d92f08ca9dccdd336a09d685c"
server_key_version: "encryptionkey@0"
I20260809 04:49:34.707572 31862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:34.712401 31862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:34.715365 31862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:34.715565 31862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:34.722728 31862 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43121
I20260809 04:49:34.722761 31934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43121 every 8 connection(s)
I20260809 04:49:34.723699 31862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:34.726504 31935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:34.728693 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 31862
I20260809 04:49:34.728844 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:34.729167 7852 external_mini_cluster.cc:1428] Setting key 13c227e24b1e6921ebe6f7ebc3379484
I20260809 04:49:34.733047 31935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009: Bootstrap starting.
I20260809 04:49:34.734884 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:34.730632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:54021 (local address 127.0.0.1:43121)
0809 04:49:34.731163 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:34.731179 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:34.731218 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:49:34.732270 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:34.732292 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:34.732596 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:34.732961 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:34.733667 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:34.733672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:34.733683 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:34.733693 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:34.733748 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:34.733766 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:34.733924 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:34.734011 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 04:49:34.734179 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":143,"threads_started":1}
I20260809 04:49:34.736006 31935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:34.736827 31935 log.cc:824] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:34.738070 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:34.738868 31935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009: No bootstrap required, opened a new log
I20260809 04:49:34.742054 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:34.740221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.143:60131 (local address 127.0.0.1:43121)
0809 04:49:34.740350 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:34.740353 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:34.740599 (+ 246us) server_negotiation.cc:407] Connection header received
0809 04:49:34.740699 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:34.740701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:34.740734 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:34.740803 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:34.741623 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:34.741626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:34.741628 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:34.741639 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:34.741664 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:34.741674 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:34.741746 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:34.741856 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 04:49:34.741891 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 04:49:34.743954 31935 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c60a0fd7584757ae7cfdf0d0a90009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43121 } }
I20260809 04:49:34.744285 31935 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:34.744347 31935 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c60a0fd7584757ae7cfdf0d0a90009, State: Initialized, Role: FOLLOWER
I20260809 04:49:34.744773 31935 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c60a0fd7584757ae7cfdf0d0a90009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43121 } }
I20260809 04:49:34.744925 31935 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:34.744992 31935 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:34.745088 31935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:34.746061 31935 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c60a0fd7584757ae7cfdf0d0a90009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43121 } }
I20260809 04:49:34.746450 31935 leader_election.cc:302] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8c60a0fd7584757ae7cfdf0d0a90009; no voters:
I20260809 04:49:34.746790 31935 leader_election.cc:288] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:34.746924 31946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:34.747171 31946 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [term 1 LEADER]: Becoming Leader. State: Replica: a8c60a0fd7584757ae7cfdf0d0a90009, State: Running, Role: LEADER
I20260809 04:49:34.747576 31946 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c60a0fd7584757ae7cfdf0d0a90009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43121 } }
I20260809 04:49:34.748252 31935 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:34.749341 31948 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8c60a0fd7584757ae7cfdf0d0a90009. Latest consensus state: current_term: 1 leader_uuid: "a8c60a0fd7584757ae7cfdf0d0a90009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c60a0fd7584757ae7cfdf0d0a90009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43121 } } }
I20260809 04:49:34.749487 31948 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:34.749809 31947 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8c60a0fd7584757ae7cfdf0d0a90009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c60a0fd7584757ae7cfdf0d0a90009" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43121 } } }
I20260809 04:49:34.749909 31955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:34.750006 31947 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:34.753343 31955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:34.758760 31955 catalog_manager.cc:1347] Generated new cluster ID: e90b842be3914464896483043a353baa
I20260809 04:49:34.758846 31955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:34.774201 31955 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:34.774545 31955 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:34.786612 31955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8c60a0fd7584757ae7cfdf0d0a90009: Generated new TSK 0
I20260809 04:49:34.787468 31955 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:34.791137 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 31862
2026-08-09T04:49:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:34 dist-test-slave-1zqn krb5kdc[31970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:34 dist-test-slave-1zqn krb5kdc[31970](info): set up 2 sockets
Aug 09 04:49:34 dist-test-slave-1zqn krb5kdc[31970](info): commencing operation
krb5kdc: starting...
W20260809 04:49:35.891191 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:35 dist-test-slave-1zqn krb5kdc[31970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:49:36.026937 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35293
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:35293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:36.142333 31986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:36.142669 31986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:36.142757 31986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:36.146878 31986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:36.146991 31986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:36.147032 31986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:36.147069 31986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:36.147109 31986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:36.152485 31986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:35293
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:36.153584 31986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:36.154659 31986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:36.163349 31994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:36.163343 31992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:36.163362 31991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:36.163750 31986 server_base.cc:1034] running on GCE node
I20260809 04:49:36.164523 31986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:36.165620 31986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:36.166795 31986 hybrid_clock.cc:648] HybridClock initialized: now 1786250976166768 us; error 38 us; skew 500 ppm
Aug 09 04:49:36 dist-test-slave-1zqn krb5kdc[31970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250976, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:36.175266 31986 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:49:36.176316 31986 webserver.cc:460] Webserver started at http://127.7.171.62:44795/ using document root <none> and password file <none>
I20260809 04:49:36.176872 31986 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:36.176932 31986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:36.177103 31986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:36.179127 31986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "4b3767ea777e41c198518a574efc75d9"
format_stamp: "Formatted at 2026-08-09 04:49:36 on dist-test-slave-1zqn"
server_key: "5987754850fcf3f2743dba394c33646f"
server_key_iv: "8cf2cde122f2bb8327f2aecada608329"
server_key_version: "encryptionkey@0"
I20260809 04:49:36.179708 31986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "4b3767ea777e41c198518a574efc75d9"
format_stamp: "Formatted at 2026-08-09 04:49:36 on dist-test-slave-1zqn"
server_key: "5987754850fcf3f2743dba394c33646f"
server_key_iv: "8cf2cde122f2bb8327f2aecada608329"
server_key_version: "encryptionkey@0"
I20260809 04:49:36.183418 31986 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260809 04:49:36.185935 32001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:36.187206 31986 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:49:36.187309 31986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "4b3767ea777e41c198518a574efc75d9"
format_stamp: "Formatted at 2026-08-09 04:49:36 on dist-test-slave-1zqn"
server_key: "5987754850fcf3f2743dba394c33646f"
server_key_iv: "8cf2cde122f2bb8327f2aecada608329"
server_key_version: "encryptionkey@0"
I20260809 04:49:36.187397 31986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:36.204103 31986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:36.207408 31986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:36.207585 31986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:36.215387 31986 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:35293
I20260809 04:49:36.215417 32056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:35293 every 8 connection(s)
I20260809 04:49:36.216408 31986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:36.219213 32057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:36.224804 32057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9: Bootstrap starting.
I20260809 04:49:36.225000 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 31986
I20260809 04:49:36.225155 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:36.225447 7852 external_mini_cluster.cc:1428] Setting key 73ad5f627ad6d9d85e17901366194e45
I20260809 04:49:36.227543 32057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:36.228363 32057 log.cc:824] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:36.230264 32057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9: No bootstrap required, opened a new log
I20260809 04:49:36.233697 32057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3767ea777e41c198518a574efc75d9" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35293 } }
I20260809 04:49:36.233940 32057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:36.234020 32057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3767ea777e41c198518a574efc75d9, State: Initialized, Role: FOLLOWER
Aug 09 04:49:36 dist-test-slave-1zqn krb5kdc[31970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:36.234544 32057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3767ea777e41c198518a574efc75d9" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35293 } }
I20260809 04:49:36.234695 32057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:36.234768 32057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:36.234901 32057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:36.235859 32057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3767ea777e41c198518a574efc75d9" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35293 } }
I20260809 04:49:36.236261 32057 leader_election.cc:302] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b3767ea777e41c198518a574efc75d9; no voters:
I20260809 04:49:36.236629 32057 leader_election.cc:288] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:36.236726 32062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:36.236954 32062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [term 1 LEADER]: Becoming Leader. State: Replica: 4b3767ea777e41c198518a574efc75d9, State: Running, Role: LEADER
I20260809 04:49:36.237335 32062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3767ea777e41c198518a574efc75d9" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35293 } }
I20260809 04:49:36.237939 32057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:36.239065 32063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b3767ea777e41c198518a574efc75d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3767ea777e41c198518a574efc75d9" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35293 } } }
I20260809 04:49:36.239210 32063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:36.240290 32072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:36.240231 32064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b3767ea777e41c198518a574efc75d9. Latest consensus state: current_term: 1 leader_uuid: "4b3767ea777e41c198518a574efc75d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b3767ea777e41c198518a574efc75d9" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35293 } } }
I20260809 04:49:36.240370 32064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:36.242762 32060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:36.226997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54927 (local address 127.7.171.62:35293)
0809 04:49:36.227428 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:36.227445 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:36.227489 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:49:36.228255 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:36.228274 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:36.228663 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:36.228995 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:36.229868 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:36.230899 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:36.231771 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:36.232332 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:36.232511 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:36.235398 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:36.235403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:36.235423 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:36.235461 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:36.237702 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:36.238354 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:36.238364 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:36.238372 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:36.238521 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:36.239046 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:36.239049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:36.239051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:36.239626 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:36.239848 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:36.241246 (+ 1398us) server_negotiation.cc:299] Negotiation successful
0809 04:49:36.241556 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":150,"threads_started":1}
I20260809 04:49:36.244252 32072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:36.247727 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:36.251801 32072 catalog_manager.cc:1347] Generated new cluster ID: cd2279fa22e14dc4afd99bdbd351bd7a
I20260809 04:49:36.251922 32072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:36.258988 32060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:36.251264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45047 (local address 127.7.171.62:35293)
0809 04:49:36.251448 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:36.251453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:36.251468 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:49:36.251506 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:36.251509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:36.251554 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:36.251647 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:36.252379 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:36.252931 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:36.253747 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:36.253999 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:36.254108 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:36.255558 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:36.255561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:36.255564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:36.255592 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:36.257197 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:36.257678 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:36.257684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:36.257691 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:36.257763 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:36.258099 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:36.258103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:36.258104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:36.258271 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:36.258450 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:36.258723 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 04:49:36.258818 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 04:49:36.262099 32072 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:36.262979 32072 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:36.268373 32072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b3767ea777e41c198518a574efc75d9: Generated new TSK 0
I20260809 04:49:36.269217 32072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:36.278260 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 31986
W20260809 04:49:36.286353 31968 connection.cc:535] client connection to 127.7.171.62:35293 recv error: Network error: failed to read from TLS socket (remote: 127.7.171.62:35293): Connection reset by peer (error 104)
2026-08-09T04:49:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1485 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:36 dist-test-slave-1zqn krb5kdc[32090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:36 dist-test-slave-1zqn krb5kdc[32090](info): set up 2 sockets
Aug 09 04:49:36 dist-test-slave-1zqn krb5kdc[32090](info): commencing operation
krb5kdc: starting...
W20260809 04:49:37.455649 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:37 dist-test-slave-1zqn krb5kdc[32090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:49:37.601552 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35177
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:43953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:35177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:37.715118 32104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:37.715471 32104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:37.715550 32104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:37.719712 32104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:37.719812 32104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:37.719853 32104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:37.719913 32104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:37.719954 32104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:37.725430 32104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:43953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:35177
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:37.726541 32104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:37.727694 32104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:37.736629 32110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:37.736728 32113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:37.736660 32111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:37.736855 32104 server_base.cc:1034] running on GCE node
I20260809 04:49:37.737470 32104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:37.738742 32104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:37.739940 32104 hybrid_clock.cc:648] HybridClock initialized: now 1786250977739921 us; error 33 us; skew 500 ppm
Aug 09 04:49:37 dist-test-slave-1zqn krb5kdc[32090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250977, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:37.743139 32104 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:49:37.744177 32104 webserver.cc:460] Webserver started at http://127.7.171.62:39693/ using document root <none> and password file <none>
I20260809 04:49:37.744771 32104 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:37.744866 32104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:37.745119 32104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:37.747275 32104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "1a0b35fb5aba4b4fb2bc988d2a2602f4"
format_stamp: "Formatted at 2026-08-09 04:49:37 on dist-test-slave-1zqn"
server_key: "eae0a774276d09a3f73c165b44f7c8ba"
server_key_iv: "498dff1120189dc755b6a736f66622e8"
server_key_version: "encryptionkey@0"
I20260809 04:49:37.747928 32104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "1a0b35fb5aba4b4fb2bc988d2a2602f4"
format_stamp: "Formatted at 2026-08-09 04:49:37 on dist-test-slave-1zqn"
server_key: "eae0a774276d09a3f73c165b44f7c8ba"
server_key_iv: "498dff1120189dc755b6a736f66622e8"
server_key_version: "encryptionkey@0"
I20260809 04:49:37.751924 32104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:49:37.754690 32120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:37.755921 32104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:49:37.756172 32104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "1a0b35fb5aba4b4fb2bc988d2a2602f4"
format_stamp: "Formatted at 2026-08-09 04:49:37 on dist-test-slave-1zqn"
server_key: "eae0a774276d09a3f73c165b44f7c8ba"
server_key_iv: "498dff1120189dc755b6a736f66622e8"
server_key_version: "encryptionkey@0"
I20260809 04:49:37.756299 32104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:37.789543 32104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:37.793226 32104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:37.793462 32104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:37.801487 32104 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:35177
I20260809 04:49:37.801504 32176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:35177 every 8 connection(s)
I20260809 04:49:37.802489 32104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:37.805402 32177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:37.809649 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 32104
I20260809 04:49:37.809767 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:37.810103 7852 external_mini_cluster.cc:1428] Setting key c0ca8d5e0d472389dd163c716edde290
I20260809 04:49:37.811825 32177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4: Bootstrap starting.
I20260809 04:49:37.814800 32177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:37.815752 32177 log.cc:824] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:37.818379 32177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4: No bootstrap required, opened a new log
I20260809 04:49:37.821593 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:37.811615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49371 (local address 127.7.171.62:35177)
0809 04:49:37.812095 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:37.812117 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:37.812166 (+ 49us) server_negotiation.cc:407] Connection header received
0809 04:49:37.813052 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:37.813082 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:37.813555 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:37.813936 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:37.814393 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:37.815944 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:37.817338 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:37.818113 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:37.818303 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:37.819285 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:37.819292 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:37.819310 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:37.819861 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:37.819932 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:37.819964 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:37.820146 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:37.820370 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 04:49:37.820533 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":113,"threads_started":1}
I20260809 04:49:37.823585 32177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0b35fb5aba4b4fb2bc988d2a2602f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35177 } }
I20260809 04:49:37.823897 32177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:37.823968 32177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a0b35fb5aba4b4fb2bc988d2a2602f4, State: Initialized, Role: FOLLOWER
I20260809 04:49:37.824505 32177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0b35fb5aba4b4fb2bc988d2a2602f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35177 } }
I20260809 04:49:37.824712 32177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:37.824818 32177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:37.824944 32177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:37.825140 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:37.826048 32177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0b35fb5aba4b4fb2bc988d2a2602f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35177 } }
I20260809 04:49:37.826462 32177 leader_election.cc:302] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a0b35fb5aba4b4fb2bc988d2a2602f4; no voters:
I20260809 04:49:37.826836 32177 leader_election.cc:288] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:37.826951 32187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:37.827265 32187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [term 1 LEADER]: Becoming Leader. State: Replica: 1a0b35fb5aba4b4fb2bc988d2a2602f4, State: Running, Role: LEADER
I20260809 04:49:37.827602 32187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0b35fb5aba4b4fb2bc988d2a2602f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35177 } }
I20260809 04:49:37.828363 32177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:37.829449 32188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a0b35fb5aba4b4fb2bc988d2a2602f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0b35fb5aba4b4fb2bc988d2a2602f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35177 } } }
I20260809 04:49:37.829638 32188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:37.830019 32189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a0b35fb5aba4b4fb2bc988d2a2602f4. Latest consensus state: current_term: 1 leader_uuid: "1a0b35fb5aba4b4fb2bc988d2a2602f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a0b35fb5aba4b4fb2bc988d2a2602f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35177 } } }
I20260809 04:49:37.830170 32189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:37.830495 32197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:37.833271 32180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:37.827357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51519 (local address 127.7.171.62:35177)
0809 04:49:37.827476 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:37.827481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:37.828014 (+ 533us) server_negotiation.cc:407] Connection header received
0809 04:49:37.828249 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:37.828253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:37.828301 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:37.828352 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:37.828791 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:37.829719 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:37.831131 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:37.831496 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:37.831563 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:37.832686 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:37.832690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:37.832693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:37.832707 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:37.832738 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:37.832751 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:37.832813 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:37.833085 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 04:49:37.833128 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 04:49:37.834154 32197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:37.839947 32197 catalog_manager.cc:1347] Generated new cluster ID: 9de8a85acd494ee7acb217ebb52ea88e
I20260809 04:49:37.840061 32197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:37.864305 32197 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:37.865617 32197 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:37.876456 32197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a0b35fb5aba4b4fb2bc988d2a2602f4: Generated new TSK 0
I20260809 04:49:37.877210 32197 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:37.888240 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 32104
W20260809 04:49:37.895792 32088 connection.cc:535] client connection to 127.7.171.62:35177 recv error: Network error: failed to read from TLS socket (remote: 127.7.171.62:35177): Connection reset by peer (error 104)
2026-08-09T04:49:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1609 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:37 dist-test-slave-1zqn krb5kdc[32211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:37 dist-test-slave-1zqn krb5kdc[32211](info): set up 2 sockets
Aug 09 04:49:37 dist-test-slave-1zqn krb5kdc[32211](info): commencing operation
krb5kdc: starting...
W20260809 04:49:39.013446 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:39 dist-test-slave-1zqn krb5kdc[32211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:49:39.152288 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46243
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:46243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:39.270869 32225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:39.271189 32225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:39.271270 32225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:39.275336 32225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:39.275444 32225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:39.275485 32225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:39.275523 32225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:39.275557 32225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:39.281044 32225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:46243
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:39.282155 32225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:39.283226 32225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:39.291970 32233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:39.291970 32231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:39.292236 32225 server_base.cc:1034] running on GCE node
W20260809 04:49:39.291978 32230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:39.292933 32225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:39.294031 32225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:39.295208 32225 hybrid_clock.cc:648] HybridClock initialized: now 1786250979295171 us; error 49 us; skew 500 ppm
Aug 09 04:49:39 dist-test-slave-1zqn krb5kdc[32211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250979, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:39.298309 32225 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:49:39.299423 32225 webserver.cc:460] Webserver started at http://127.7.171.62:35513/ using document root <none> and password file <none>
I20260809 04:49:39.299979 32225 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:39.300040 32225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:39.300287 32225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:39.302352 32225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "c7ef7576a9b943189b6a49d5f7c87715"
format_stamp: "Formatted at 2026-08-09 04:49:39 on dist-test-slave-1zqn"
server_key: "fd5971852ae7f26493636b2a6d9fea0e"
server_key_iv: "d2aa6ad4dd03bffe089cc8be423982c1"
server_key_version: "encryptionkey@0"
I20260809 04:49:39.302865 32225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "c7ef7576a9b943189b6a49d5f7c87715"
format_stamp: "Formatted at 2026-08-09 04:49:39 on dist-test-slave-1zqn"
server_key: "fd5971852ae7f26493636b2a6d9fea0e"
server_key_iv: "d2aa6ad4dd03bffe089cc8be423982c1"
server_key_version: "encryptionkey@0"
I20260809 04:49:39.306692 32225 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 04:49:39.309438 32240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:39.310961 32225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:49:39.311218 32225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "c7ef7576a9b943189b6a49d5f7c87715"
format_stamp: "Formatted at 2026-08-09 04:49:39 on dist-test-slave-1zqn"
server_key: "fd5971852ae7f26493636b2a6d9fea0e"
server_key_iv: "d2aa6ad4dd03bffe089cc8be423982c1"
server_key_version: "encryptionkey@0"
I20260809 04:49:39.311357 32225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:39.316581 32225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:39.320358 32225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:39.320568 32225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:39.328665 32225 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:46243
I20260809 04:49:39.328678 32297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:46243 every 8 connection(s)
I20260809 04:49:39.329748 32225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:39.332713 32298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:39.338263 32298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715: Bootstrap starting.
I20260809 04:49:39.340016 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 32225
I20260809 04:49:39.340271 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:39.340586 7852 external_mini_cluster.cc:1428] Setting key d7735baf00cdd84eb949410047b5c024
I20260809 04:49:39.340896 32298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:39.341830 32298 log.cc:824] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:39.343791 32298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715: No bootstrap required, opened a new log
I20260809 04:49:39.345889 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:39.341804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33517 (local address 127.7.171.62:46243)
0809 04:49:39.342309 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:39.342326 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:39.342363 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:49:39.343050 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:39.343068 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:39.343357 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:39.343656 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:39.344514 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:39.344520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:39.344532 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:39.344542 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:39.344596 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:39.344616 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:39.344707 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:39.344858 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 04:49:39.345154 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":192,"threads_started":1}
I20260809 04:49:39.348047 32298 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ef7576a9b943189b6a49d5f7c87715" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46243 } }
I20260809 04:49:39.348376 32298 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:39.348474 32298 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ef7576a9b943189b6a49d5f7c87715, State: Initialized, Role: FOLLOWER
I20260809 04:49:39.348989 32298 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ef7576a9b943189b6a49d5f7c87715" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46243 } }
I20260809 04:49:39.349151 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:39.349164 32298 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:39.349229 32298 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:39.349323 32298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:39.350279 32298 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ef7576a9b943189b6a49d5f7c87715" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46243 } }
I20260809 04:49:39.350652 32298 leader_election.cc:302] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7ef7576a9b943189b6a49d5f7c87715; no voters:
I20260809 04:49:39.351023 32298 leader_election.cc:288] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:39.351089 32308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:39.351305 32308 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [term 1 LEADER]: Becoming Leader. State: Replica: c7ef7576a9b943189b6a49d5f7c87715, State: Running, Role: LEADER
I20260809 04:49:39.351649 32308 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ef7576a9b943189b6a49d5f7c87715" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46243 } }
I20260809 04:49:39.352349 32298 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:39.352869 32310 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7ef7576a9b943189b6a49d5f7c87715. Latest consensus state: current_term: 1 leader_uuid: "c7ef7576a9b943189b6a49d5f7c87715" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ef7576a9b943189b6a49d5f7c87715" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46243 } } }
I20260809 04:49:39.353055 32310 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:39.353379 32309 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7ef7576a9b943189b6a49d5f7c87715" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ef7576a9b943189b6a49d5f7c87715" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46243 } } }
I20260809 04:49:39.353571 32309 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:39.354168 32314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:39.354532 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:39.351749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35535 (local address 127.7.171.62:46243)
0809 04:49:39.351862 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:39.351869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:39.352304 (+ 435us) server_negotiation.cc:407] Connection header received
0809 04:49:39.352494 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:39.352497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:39.352532 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:39.352601 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:39.353967 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:39.353971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:39.353973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:39.353988 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:39.354017 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:39.354035 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:39.354136 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:39.354323 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 04:49:39.354368 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 04:49:39.357599 32314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:39.363653 32314 catalog_manager.cc:1347] Generated new cluster ID: 95d3f1124f7444faa3e8fece326f1f87
I20260809 04:49:39.363729 32314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:39.377995 32314 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:39.378403 32314 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:39.382925 32314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7ef7576a9b943189b6a49d5f7c87715: Generated new TSK 0
I20260809 04:49:39.383692 32314 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:39.391319 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 32225
2026-08-09T04:49:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:39 dist-test-slave-1zqn krb5kdc[32330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:39 dist-test-slave-1zqn krb5kdc[32330](info): set up 2 sockets
Aug 09 04:49:39 dist-test-slave-1zqn krb5kdc[32330](info): commencing operation
krb5kdc: starting...
W20260809 04:49:40.579367 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:40 dist-test-slave-1zqn krb5kdc[32330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:49:40.720270 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:39839
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:33469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:39839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:40.836480 32345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:40.836752 32345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:40.836824 32345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:40.840911 32345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:40.840978 32345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:40.840997 32345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:40.841054 32345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:40.841118 32345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:40.846496 32345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:33469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:39839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:39839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:40.847604 32345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:40.848678 32345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:40.857867 32351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:40.858058 32345 server_base.cc:1034] running on GCE node
W20260809 04:49:40.857944 32354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:40.857846 32352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:40.858821 32345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:40.859992 32345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:40.861210 32345 hybrid_clock.cc:648] HybridClock initialized: now 1786250980861179 us; error 46 us; skew 500 ppm
Aug 09 04:49:40 dist-test-slave-1zqn krb5kdc[32330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250980, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:40.864023 32345 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:49:40.865017 32345 webserver.cc:460] Webserver started at http://127.7.171.62:37717/ using document root <none> and password file <none>
I20260809 04:49:40.865607 32345 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:40.865702 32345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:40.865942 32345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:40.868031 32345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "1aeb6340b0ba4094895c27c8a2b62cf4"
format_stamp: "Formatted at 2026-08-09 04:49:40 on dist-test-slave-1zqn"
server_key: "57c68d45d91c47d2f28601a64f8335bc"
server_key_iv: "9e2071dcf5fadb3dbd1f3a12695853b8"
server_key_version: "encryptionkey@0"
I20260809 04:49:40.868708 32345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "1aeb6340b0ba4094895c27c8a2b62cf4"
format_stamp: "Formatted at 2026-08-09 04:49:40 on dist-test-slave-1zqn"
server_key: "57c68d45d91c47d2f28601a64f8335bc"
server_key_iv: "9e2071dcf5fadb3dbd1f3a12695853b8"
server_key_version: "encryptionkey@0"
I20260809 04:49:40.872635 32345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:49:40.875211 32361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:40.876466 32345 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:40.876626 32345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "1aeb6340b0ba4094895c27c8a2b62cf4"
format_stamp: "Formatted at 2026-08-09 04:49:40 on dist-test-slave-1zqn"
server_key: "57c68d45d91c47d2f28601a64f8335bc"
server_key_iv: "9e2071dcf5fadb3dbd1f3a12695853b8"
server_key_version: "encryptionkey@0"
I20260809 04:49:40.876742 32345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:40.902032 32345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:40.905366 32345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:40.905613 32345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:40.913370 32345 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:39839
I20260809 04:49:40.913395 32418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:39839 every 8 connection(s)
I20260809 04:49:40.914395 32345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:40.917318 32419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:40.918339 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 32345
I20260809 04:49:40.918500 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:40.918860 7852 external_mini_cluster.cc:1428] Setting key 7deca76ff3366df8d8ac2b8c65a91f96
I20260809 04:49:40.923632 32419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4: Bootstrap starting.
I20260809 04:49:40.925966 32419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:40.926707 32419 log.cc:824] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:49:40 dist-test-slave-1zqn krb5kdc[32330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:40.928951 32419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4: No bootstrap required, opened a new log
I20260809 04:49:40.933765 32422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:40.920348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48841 (local address 127.7.171.62:39839)
0809 04:49:40.920782 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:40.920797 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:40.920834 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:49:40.921501 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:40.921519 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:40.921863 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:40.922210 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:40.923083 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:40.924171 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:40.925069 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:40.925544 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:40.925722 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:40.928494 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:40.928501 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:40.928516 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:40.928548 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:40.930547 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:40.930973 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:40.930979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:40.930985 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:40.931142 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:40.931412 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:40.931417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:40.931419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:40.931994 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:40.932227 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:40.932462 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 04:49:40.932768 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":144,"threads_started":1}
I20260809 04:49:40.933892 32419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb6340b0ba4094895c27c8a2b62cf4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39839 } }
I20260809 04:49:40.934154 32419 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:40.934211 32419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aeb6340b0ba4094895c27c8a2b62cf4, State: Initialized, Role: FOLLOWER
I20260809 04:49:40.934679 32419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb6340b0ba4094895c27c8a2b62cf4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39839 } }
I20260809 04:49:40.934849 32419 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:40.934926 32419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:40.935075 32419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:40.936311 32419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb6340b0ba4094895c27c8a2b62cf4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39839 } }
I20260809 04:49:40.936718 32419 leader_election.cc:302] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aeb6340b0ba4094895c27c8a2b62cf4; no voters:
I20260809 04:49:40.936893 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:40.937084 32419 leader_election.cc:288] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:40.937186 32424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:40.937366 32424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [term 1 LEADER]: Becoming Leader. State: Replica: 1aeb6340b0ba4094895c27c8a2b62cf4, State: Running, Role: LEADER
I20260809 04:49:40.937738 32424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb6340b0ba4094895c27c8a2b62cf4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39839 } }
I20260809 04:49:40.938468 32419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:40.939064 32424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aeb6340b0ba4094895c27c8a2b62cf4. Latest consensus state: current_term: 1 leader_uuid: "1aeb6340b0ba4094895c27c8a2b62cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb6340b0ba4094895c27c8a2b62cf4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39839 } } }
I20260809 04:49:40.939185 32424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:40.939489 32425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aeb6340b0ba4094895c27c8a2b62cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aeb6340b0ba4094895c27c8a2b62cf4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 39839 } } }
I20260809 04:49:40.939599 32425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4 [sys.catalog]: This master's current role is: LEADER
W20260809 04:49:40.943409 32445 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:40.943496 32445 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:40.943567 32436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:40.944407 32436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:40.949785 32422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:40.939547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42609 (local address 127.7.171.62:39839)
0809 04:49:40.939665 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:40.939671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:40.941197 (+ 1526us) server_negotiation.cc:407] Connection header received
0809 04:49:40.941524 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:40.941527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:40.941580 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:40.941674 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:40.942512 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:40.943303 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:40.944347 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:40.944652 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:40.944768 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:40.946150 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:40.946154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:40.946157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:40.946182 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:40.947912 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:40.948470 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:40.948473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:40.948474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:40.948560 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:40.948864 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:40.948868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:40.948869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:40.949055 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:40.949205 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:40.949454 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 04:49:40.949587 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:49:40.950981 32436 catalog_manager.cc:1347] Generated new cluster ID: 95e7ec1572254903ab71e9091fa48f71
I20260809 04:49:40.951051 32436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:41.000803 32436 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:41.002183 32436 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:41.010835 32436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aeb6340b0ba4094895c27c8a2b62cf4: Generated new TSK 0
I20260809 04:49:41.011711 32436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:41.023630 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 32345
2026-08-09T04:49:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1632 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:41 dist-test-slave-1zqn krb5kdc[32451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:41 dist-test-slave-1zqn krb5kdc[32451](info): set up 2 sockets
Aug 09 04:49:41 dist-test-slave-1zqn krb5kdc[32451](info): commencing operation
krb5kdc: starting...
W20260809 04:49:42.132499 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:42 dist-test-slave-1zqn krb5kdc[32451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:49:42.275794 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46737
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:46737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:42.390407 32467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:42.390700 32467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:42.390743 32467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:42.394901 32467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:42.395051 32467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:42.395107 32467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:42.395130 32467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:42.395184 32467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:42.400848 32467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:46737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:42.402067 32467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:42.403415 32467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:42.412387 32472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:42.412393 32475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:42.412382 32473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:42.412444 32467 server_base.cc:1034] running on GCE node
I20260809 04:49:42.413322 32467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:42.414505 32467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:42.415720 32467 hybrid_clock.cc:648] HybridClock initialized: now 1786250982415703 us; error 33 us; skew 500 ppm
Aug 09 04:49:42 dist-test-slave-1zqn krb5kdc[32451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250982, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:42.419025 32467 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:49:42.419958 32467 webserver.cc:460] Webserver started at http://127.7.171.62:44215/ using document root <none> and password file <none>
I20260809 04:49:42.420583 32467 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:42.420653 32467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:42.420830 32467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:42.423002 32467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "1fedd7b6bb8f447f8cd78207326e872e"
format_stamp: "Formatted at 2026-08-09 04:49:42 on dist-test-slave-1zqn"
server_key: "175d80571e675f6087ffbce5b2433112"
server_key_iv: "f7501147c5bb58f75a4ec9d7dd76be34"
server_key_version: "encryptionkey@0"
I20260809 04:49:42.423557 32467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "1fedd7b6bb8f447f8cd78207326e872e"
format_stamp: "Formatted at 2026-08-09 04:49:42 on dist-test-slave-1zqn"
server_key: "175d80571e675f6087ffbce5b2433112"
server_key_iv: "f7501147c5bb58f75a4ec9d7dd76be34"
server_key_version: "encryptionkey@0"
I20260809 04:49:42.427650 32467 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:49:42.430423 32482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:42.431852 32467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:49:42.432014 32467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "1fedd7b6bb8f447f8cd78207326e872e"
format_stamp: "Formatted at 2026-08-09 04:49:42 on dist-test-slave-1zqn"
server_key: "175d80571e675f6087ffbce5b2433112"
server_key_iv: "f7501147c5bb58f75a4ec9d7dd76be34"
server_key_version: "encryptionkey@0"
I20260809 04:49:42.432194 32467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:42.478221 32467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:42.481827 32467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:42.482066 32467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:42.489883 32467 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:46737
I20260809 04:49:42.489915 32538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:46737 every 8 connection(s)
I20260809 04:49:42.490895 32467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:42.493754 32539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:42.494148 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 32467
I20260809 04:49:42.494299 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:42.494625 7852 external_mini_cluster.cc:1428] Setting key 3d77aa7d344d754aadd596cf98691b38
I20260809 04:49:42.500628 32539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e: Bootstrap starting.
I20260809 04:49:42.503490 32539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:42.504508 32539 log.cc:824] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:42.505535 32542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:42.496196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36889 (local address 127.7.171.62:46737)
0809 04:49:42.496674 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:42.496694 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:42.496740 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:49:42.497583 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:42.497606 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:42.498042 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:42.498396 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:42.498816 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:42.500344 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:42.501501 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:42.502377 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:42.502546 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:42.503454 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:42.503461 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:42.503480 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:42.504021 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:42.504123 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:42.504154 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:42.504250 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:42.504436 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 04:49:42.504665 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":152,"threads_started":1}
I20260809 04:49:42.507015 32539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e: No bootstrap required, opened a new log
I20260809 04:49:42.509567 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:42.511332 32539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fedd7b6bb8f447f8cd78207326e872e" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46737 } }
I20260809 04:49:42.511600 32539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:42.511729 32539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fedd7b6bb8f447f8cd78207326e872e, State: Initialized, Role: FOLLOWER
I20260809 04:49:42.512233 32539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fedd7b6bb8f447f8cd78207326e872e" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46737 } }
I20260809 04:49:42.512380 32539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:42.512473 32539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:42.512583 32539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:42.513489 32539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fedd7b6bb8f447f8cd78207326e872e" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46737 } }
I20260809 04:49:42.513821 32539 leader_election.cc:302] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fedd7b6bb8f447f8cd78207326e872e; no voters:
I20260809 04:49:42.514151 32539 leader_election.cc:288] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:42.514211 32549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:42.514430 32549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [term 1 LEADER]: Becoming Leader. State: Replica: 1fedd7b6bb8f447f8cd78207326e872e, State: Running, Role: LEADER
I20260809 04:49:42.514725 32549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fedd7b6bb8f447f8cd78207326e872e" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46737 } }
I20260809 04:49:42.515460 32539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:42.515899 32549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fedd7b6bb8f447f8cd78207326e872e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fedd7b6bb8f447f8cd78207326e872e" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46737 } } }
I20260809 04:49:42.516029 32549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:42.516261 32549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fedd7b6bb8f447f8cd78207326e872e. Latest consensus state: current_term: 1 leader_uuid: "1fedd7b6bb8f447f8cd78207326e872e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fedd7b6bb8f447f8cd78207326e872e" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46737 } } }
I20260809 04:49:42.516775 32549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:42.516417 32557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:42.517416 32542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:42.511574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47749 (local address 127.7.171.62:46737)
0809 04:49:42.511696 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:42.511701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:42.511863 (+ 162us) server_negotiation.cc:407] Connection header received
0809 04:49:42.512004 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:42.512007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:42.512105 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:42.512173 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:42.512466 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:42.513049 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:42.513911 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:42.514301 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:42.514398 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:42.516938 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:42.516941 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:42.516944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:42.516959 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:42.516986 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:42.516997 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:42.517052 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:42.517234 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 04:49:42.517291 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:49:42.520247 32557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:42.526096 32557 catalog_manager.cc:1347] Generated new cluster ID: 0fc6d86821604235b6410471025ef481
I20260809 04:49:42.526172 32557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:42.547426 32557 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:42.548813 32557 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:42.560187 32557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fedd7b6bb8f447f8cd78207326e872e: Generated new TSK 0
I20260809 04:49:42.561223 32557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:42.568584 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 32467
2026-08-09T04:49:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1543 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:42 dist-test-slave-1zqn krb5kdc[32574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:42 dist-test-slave-1zqn krb5kdc[32574](info): set up 2 sockets
Aug 09 04:49:42 dist-test-slave-1zqn krb5kdc[32574](info): commencing operation
krb5kdc: starting...
W20260809 04:49:43.709614 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:43 dist-test-slave-1zqn krb5kdc[32574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:49:43.851522 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45085
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:44281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:45085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:43.969834 32588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:43.970181 32588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:43.970271 32588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:43.974426 32588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:43.974514 32588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:43.974555 32588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:43.974613 32588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:43.974655 32588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:43.979984 32588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:44281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:45085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:45085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:43.981132 32588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:43.982206 32588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:43.990942 32596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:43.991170 32594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:43.991393 32588 server_base.cc:1034] running on GCE node
W20260809 04:49:43.991170 32593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:43.992158 32588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:43.993224 32588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:43.994421 32588 hybrid_clock.cc:648] HybridClock initialized: now 1786250983994406 us; error 28 us; skew 500 ppm
Aug 09 04:49:43 dist-test-slave-1zqn krb5kdc[32574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250983, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:43.997349 32588 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:49:43.998227 32588 webserver.cc:460] Webserver started at http://127.7.171.62:40967/ using document root <none> and password file <none>
I20260809 04:49:43.998757 32588 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:43.998819 32588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:43.999045 32588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:44.001210 32588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "d20e5fe3efa948bb8234e9a320f1d81b"
format_stamp: "Formatted at 2026-08-09 04:49:43 on dist-test-slave-1zqn"
server_key: "c0cd17a84096e01b686453edec1f9037"
server_key_iv: "1f310d883a48a9bce332cf2dfdae578d"
server_key_version: "encryptionkey@0"
I20260809 04:49:44.001736 32588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "d20e5fe3efa948bb8234e9a320f1d81b"
format_stamp: "Formatted at 2026-08-09 04:49:43 on dist-test-slave-1zqn"
server_key: "c0cd17a84096e01b686453edec1f9037"
server_key_iv: "1f310d883a48a9bce332cf2dfdae578d"
server_key_version: "encryptionkey@0"
I20260809 04:49:44.005537 32588 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:49:44.008278 32603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:44.009609 32588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:44.009759 32588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "d20e5fe3efa948bb8234e9a320f1d81b"
format_stamp: "Formatted at 2026-08-09 04:49:43 on dist-test-slave-1zqn"
server_key: "c0cd17a84096e01b686453edec1f9037"
server_key_iv: "1f310d883a48a9bce332cf2dfdae578d"
server_key_version: "encryptionkey@0"
I20260809 04:49:44.009877 32588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:44.014797 32588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:44.018225 32588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:44.018446 32588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:44.026268 32588 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:45085
I20260809 04:49:44.026297 32659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:45085 every 8 connection(s)
I20260809 04:49:44.027233 32588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:44.029243 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 32588
I20260809 04:49:44.029394 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:44.029722 7852 external_mini_cluster.cc:1428] Setting key eae73d826abcca31424e79c7c635ba1d
I20260809 04:49:44.030503 32660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:44.035369 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:44.031184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47413 (local address 127.7.171.62:45085)
0809 04:49:44.031631 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:44.031650 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:44.031693 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:49:44.032574 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:44.032598 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:44.032958 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:44.033324 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:44.033999 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:44.034005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:44.034019 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:44.034030 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:44.034093 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:44.034121 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:44.034253 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:44.034329 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 04:49:44.034491 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":138,"threads_started":1}
I20260809 04:49:44.037645 32660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b: Bootstrap starting.
I20260809 04:49:44.038368 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:44.040494 32660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:44.041432 32660 log.cc:824] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:44.042636 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:44.040733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37533 (local address 127.7.171.62:45085)
0809 04:49:44.040858 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:44.040863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:44.041050 (+ 187us) server_negotiation.cc:407] Connection header received
0809 04:49:44.041160 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:44.041163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:44.041201 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:44.041284 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:44.042045 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:44.042049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:44.042052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:44.042065 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:44.042094 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:44.042110 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:44.042201 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:44.042405 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 04:49:44.042446 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:49:44.043849 32660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b: No bootstrap required, opened a new log
I20260809 04:49:44.047575 32660 raft_consensus.cc:348] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d20e5fe3efa948bb8234e9a320f1d81b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45085 } }
I20260809 04:49:44.047842 32660 raft_consensus.cc:374] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:44.047931 32660 raft_consensus.cc:729] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d20e5fe3efa948bb8234e9a320f1d81b, State: Initialized, Role: FOLLOWER
I20260809 04:49:44.048406 32660 consensus_queue.cc:260] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d20e5fe3efa948bb8234e9a320f1d81b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45085 } }
I20260809 04:49:44.048543 32660 raft_consensus.cc:388] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:44.048590 32660 raft_consensus.cc:482] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:44.048667 32660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:44.049933 32660 raft_consensus.cc:504] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d20e5fe3efa948bb8234e9a320f1d81b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45085 } }
I20260809 04:49:44.050251 32660 leader_election.cc:302] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d20e5fe3efa948bb8234e9a320f1d81b; no voters:
I20260809 04:49:44.050542 32660 leader_election.cc:288] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:44.050669 32670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:44.050941 32670 raft_consensus.cc:686] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [term 1 LEADER]: Becoming Leader. State: Replica: d20e5fe3efa948bb8234e9a320f1d81b, State: Running, Role: LEADER
I20260809 04:49:44.051293 32670 consensus_queue.cc:237] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d20e5fe3efa948bb8234e9a320f1d81b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45085 } }
I20260809 04:49:44.051826 32660 sys_catalog.cc:564] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:44.053103 32672 sys_catalog.cc:455] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d20e5fe3efa948bb8234e9a320f1d81b. Latest consensus state: current_term: 1 leader_uuid: "d20e5fe3efa948bb8234e9a320f1d81b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d20e5fe3efa948bb8234e9a320f1d81b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45085 } } }
I20260809 04:49:44.053085 32671 sys_catalog.cc:455] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d20e5fe3efa948bb8234e9a320f1d81b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d20e5fe3efa948bb8234e9a320f1d81b" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 45085 } } }
I20260809 04:49:44.053234 32672 sys_catalog.cc:458] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:44.053234 32671 sys_catalog.cc:458] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:44.054220 32679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:44.057600 32679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:44.063612 32679 catalog_manager.cc:1347] Generated new cluster ID: d2860792955a4a698b005cf5400a18e5
I20260809 04:49:44.063767 32679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:44.094519 32679 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:44.094909 32679 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:44.101704 32679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d20e5fe3efa948bb8234e9a320f1d81b: Generated new TSK 0
I20260809 04:49:44.102718 32679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:44.112398 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 32588
2026-08-09T04:49:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1545 ms)
[----------] 18 tests from AuthTokenIssuingTest (27715 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:44 dist-test-slave-1zqn krb5kdc[32692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:44 dist-test-slave-1zqn krb5kdc[32692](info): set up 2 sockets
Aug 09 04:49:44 dist-test-slave-1zqn krb5kdc[32692](info): commencing operation
krb5kdc: starting...
W20260809 04:49:45.259841 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.006s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:45 dist-test-slave-1zqn krb5kdc[32692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:49:45.397257 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35105
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:35105,127.7.171.61:46161,127.7.171.60:45837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:45.510747 32708 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:45.511036 32708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:45.511070 32708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:45.511162 32708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:45.515166 32708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:45.515228 32708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:45.515249 32708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:45.515264 32708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:45.515319 32708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:45.521528 32708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:35105,127.7.171.61:46161,127.7.171.60:45837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:35105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:45.522579 32708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:45.523594 32708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:45.532526 32714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:45.532527 32713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:45.532606 32708 server_base.cc:1034] running on GCE node
W20260809 04:49:45.532548 32716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:45.533288 32708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:45.534438 32708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:45.535631 32708 hybrid_clock.cc:648] HybridClock initialized: now 1786250985535616 us; error 27 us; skew 500 ppm
Aug 09 04:49:45 dist-test-slave-1zqn krb5kdc[32692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:45.538532 32708 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:49:45.539364 32708 webserver.cc:460] Webserver started at http://127.7.171.62:36575/ using document root <none> and password file <none>
I20260809 04:49:45.539889 32708 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:45.539953 32708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:45.540177 32708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:45.542171 32708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "b85bd9e698c34e57bf8944fcaf97a227"
format_stamp: "Formatted at 2026-08-09 04:49:45 on dist-test-slave-1zqn"
server_key: "8ac974f0e21f36aaad14b998577be507"
server_key_iv: "e3136a06484366dfc900ba2f78518d56"
server_key_version: "encryptionkey@0"
I20260809 04:49:45.542678 32708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "b85bd9e698c34e57bf8944fcaf97a227"
format_stamp: "Formatted at 2026-08-09 04:49:45 on dist-test-slave-1zqn"
server_key: "8ac974f0e21f36aaad14b998577be507"
server_key_iv: "e3136a06484366dfc900ba2f78518d56"
server_key_version: "encryptionkey@0"
I20260809 04:49:45.546473 32708 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:49:45.549057 32723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:45.550305 32708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 04:49:45.550437 32708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "b85bd9e698c34e57bf8944fcaf97a227"
format_stamp: "Formatted at 2026-08-09 04:49:45 on dist-test-slave-1zqn"
server_key: "8ac974f0e21f36aaad14b998577be507"
server_key_iv: "e3136a06484366dfc900ba2f78518d56"
server_key_version: "encryptionkey@0"
I20260809 04:49:45.550542 32708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:45.563706 32708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:45.567152 32708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:45.567360 32708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:45.574542 32708 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:35105
I20260809 04:49:45.574849 309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:35105 every 8 connection(s)
I20260809 04:49:45.575527 32708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:45.578333 310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:45.582029 310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:45.584493 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 32708
I20260809 04:49:45.584636 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:45.584924 7852 external_mini_cluster.cc:1428] Setting key a0e35edac8351c80873e93b27d51cf2d
Aug 09 04:49:45 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.171.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.61@KRBTEST.COM" created.
I20260809 04:49:45.597025 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.584295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.7.171.62:35105)
0809 04:49:45.584871 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:45.584902 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:45.585116 (+ 214us) server_negotiation.cc:407] Connection header received
0809 04:49:45.586152 (+ 1036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:45.586158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:45.586339 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:45.586532 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:45.587390 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.588311 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.589287 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.589660 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.589891 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:45.592250 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:45.592255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:45.592265 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:45.592285 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:45.593894 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:45.594433 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:45.594439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:45.594441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:45.594552 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:45.594863 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:45.594866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:45.594867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:45.595250 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:45.595419 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:45.595764 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 04:49:45.596073 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":437,"thread_start_us":154,"threads_started":1}
I20260809 04:49:45.597030 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.584291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35105 (local address 127.0.0.1:46552)
0809 04:49:45.584921 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.584999 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:45.585849 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:45.586526 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:45.586531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:45.586994 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:45.587240 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:45.587254 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.588428 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.588434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:45.589166 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:45.589174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.589815 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.589819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:45.589981 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:45.590578 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:45.590594 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:45.592098 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:45.594025 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:45.594035 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:45.594045 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:45.594323 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:45.594645 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:45.594650 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:45.594652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:45.594767 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:45.595369 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:45.595376 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:45.595526 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 04:49:45.595699 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 04:49:45.596195 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":193,"spinlock_wait_cycles":7040,"thread_start_us":153,"threads_started":1}
I20260809 04:49:45.599431 310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:45.600342 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.599934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:46161 (local address 127.0.0.1:44550)
0809 04:49:45.600160 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.600254 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260809 04:49:45.600605 32724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
I20260809 04:49:45.601653 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.601367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:46161 (local address 127.0.0.1:44564)
0809 04:49:45.601517 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.601580 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 04:49:45.601804 310 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.61:46161: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab.
Entry for principal kudu/127.7.171.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab.
I20260809 04:49:45.616612 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.61:46161
--webserver_interface=127.7.171.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab
--principal=kudu/127.7.171.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:35105,127.7.171.61:46161,127.7.171.60:45837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:49:45.651093 310 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } attempt: 1
I20260809 04:49:45.652201 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.651803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:46161 (local address 127.0.0.1:44566)
0809 04:49:45.651998 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.652098 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 04:49:45.653064 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.652786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:46161 (local address 127.0.0.1:44572)
0809 04:49:45.652906 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.652968 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:49:45.653259 310 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.61:46161: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
I20260809 04:49:45.721467 310 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } attempt: 2
I20260809 04:49:45.722362 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.722021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:46161 (local address 127.0.0.1:44576)
0809 04:49:45.722215 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.722283 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 04:49:45.723188 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.722915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:46161 (local address 127.0.0.1:44590)
0809 04:49:45.723064 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.723131 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260809 04:49:45.723359 310 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.61:46161: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
W20260809 04:49:45.733608 317 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:45.733960 317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:45.734020 317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:45.734108 317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:45.738329 317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:45.738416 317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:45.738445 317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:45.738476 317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:45.738526 317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:45.744181 317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:35105,127.7.171.61:46161,127.7.171.60:45837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab
--principal=kudu/127.7.171.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.61:46161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.171.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:45.745353 317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:45.746418 317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:45.755128 323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:45.755088 322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:45.755138 325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:45.755333 317 server_base.cc:1034] running on GCE node
I20260809 04:49:45.756172 317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:45.757295 317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:45.758580 317 hybrid_clock.cc:648] HybridClock initialized: now 1786250985758543 us; error 46 us; skew 500 ppm
Aug 09 04:49:45 dist-test-slave-1zqn krb5kdc[32692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:45.761857 317 init.cc:377] Logged in from keytab as kudu/127.7.171.61@KRBTEST.COM (short username kudu)
I20260809 04:49:45.763049 317 webserver.cc:460] Webserver started at http://127.7.171.61:38053/ using document root <none> and password file <none>
I20260809 04:49:45.763690 317 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:45.763798 317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:45.764127 317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:45.766480 317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/data/instance:
uuid: "9a02fb34b4894d54bffd7dccc326ba17"
format_stamp: "Formatted at 2026-08-09 04:49:45 on dist-test-slave-1zqn"
server_key: "eee13088691c9c4cee72c9d4d972d6cf"
server_key_iv: "19906f15ddaf5283bddf48920805bda0"
server_key_version: "encryptionkey@0"
I20260809 04:49:45.767150 317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/wal/instance:
uuid: "9a02fb34b4894d54bffd7dccc326ba17"
format_stamp: "Formatted at 2026-08-09 04:49:45 on dist-test-slave-1zqn"
server_key: "eee13088691c9c4cee72c9d4d972d6cf"
server_key_iv: "19906f15ddaf5283bddf48920805bda0"
server_key_version: "encryptionkey@0"
I20260809 04:49:45.771482 317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:49:45.773989 332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:45.775256 317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:49:45.775420 317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/wal
uuid: "9a02fb34b4894d54bffd7dccc326ba17"
format_stamp: "Formatted at 2026-08-09 04:49:45 on dist-test-slave-1zqn"
server_key: "eee13088691c9c4cee72c9d4d972d6cf"
server_key_iv: "19906f15ddaf5283bddf48920805bda0"
server_key_version: "encryptionkey@0"
I20260809 04:49:45.775544 317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:45.810189 317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:45.814031 317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:45.814273 317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:45.814529 310 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } attempt: 3
I20260809 04:49:45.815490 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.815110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:46161 (local address 127.0.0.1:44604)
0809 04:49:45.815294 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.815376 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 04:49:45.816321 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.816105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:46161 (local address 127.0.0.1:44618)
0809 04:49:45.816203 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.816258 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 04:49:45.816531 310 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.61:46161: Network error: Client connection negotiation failed: client connection to 127.7.171.61:46161: connect: Connection refused (error 111)
I20260809 04:49:45.822830 317 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.61:46161
I20260809 04:49:45.823230 386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.61:46161 every 8 connection(s)
I20260809 04:49:45.823894 317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
I20260809 04:49:45.824820 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 317
I20260809 04:49:45.824930 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-1/wal/instance
I20260809 04:49:45.825244 7852 external_mini_cluster.cc:1428] Setting key c4cb1aa24336b666c458e3fef358fce5
I20260809 04:49:45.827188 387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:45.831374 387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.171.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.60@KRBTEST.COM" created.
Aug 09 04:49:45 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:45.848328 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.833596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46566 (local address 127.7.171.62:35105)
0809 04:49:45.833779 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:45.833785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:45.834944 (+ 1159us) server_negotiation.cc:407] Connection header received
0809 04:49:45.836452 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:45.836457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:45.836537 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:45.836651 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:45.838113 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.838752 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.840113 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.840418 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.840560 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:45.843678 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:45.843682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:45.843684 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:45.843715 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:45.845602 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:45.846333 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:45.846338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:45.846340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:45.846450 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:45.846899 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:45.846904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:45.846906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:45.847147 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:45.847276 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:45.847920 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 04:49:45.848109 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:49:45.849243 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.834094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35105 (local address 127.0.0.1:46566)
0809 04:49:45.834723 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.834792 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:45.836182 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:45.836944 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:45.836956 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:45.837360 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:45.837927 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:45.837942 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.838913 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.838917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:45.839942 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:45.839953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.840608 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.840613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:45.840821 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:45.841543 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:45.841563 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:45.843460 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:45.845783 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:45.845790 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:45.845809 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:45.846210 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:45.846570 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:45.846574 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:45.846576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:45.846764 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:45.847276 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:45.847285 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:45.847604 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 04:49:45.847840 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 04:49:45.848261 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":120,"threads_started":1}
I20260809 04:49:45.850894 387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab.
Entry for principal kudu/127.7.171.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab.
I20260809 04:49:45.856235 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.60:45837
--webserver_interface=127.7.171.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab
--principal=kudu/127.7.171.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:42369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:35105,127.7.171.61:46161,127.7.171.60:45837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:49:45 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
I20260809 04:49:45.862390 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.851571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:46161 (local address 127.0.0.1:44630)
0809 04:49:45.851716 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.851729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:45.851826 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:45.852640 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:45.852643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:45.852841 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:45.852941 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:45.852946 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.854048 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.854068 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:45.854700 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:45.854724 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.855509 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.855512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:45.855599 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:45.856184 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:45.856196 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:45.857803 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:45.860653 (+ 2850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:45.860655 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:45.860657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:45.860854 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:45.861217 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:45.861220 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:45.861221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:45.861301 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:45.861970 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:45.861972 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:45.862064 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 04:49:45.862151 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 04:49:45.862260 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 04:49:45.862979 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.851704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.7.171.61:46161)
0809 04:49:45.852106 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:45.852129 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:45.852160 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:49:45.852203 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:45.852207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:45.852393 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:45.852518 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:45.853068 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.853911 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.854835 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.855353 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.855554 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:45.857968 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:45.857981 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:45.858036 (+ 55us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:45.858091 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:45.860517 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:45.860984 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:45.860995 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:45.861000 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:45.861114 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:45.861402 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:45.861417 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:45.861423 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:45.861841 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:45.862013 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:45.862309 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 04:49:45.862702 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":141,"threads_started":1}
I20260809 04:49:45.864990 387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:45.865782 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.865542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38814)
0809 04:49:45.865660 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.865726 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 04:49:45.865998 333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
I20260809 04:49:45.867013 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.866764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38828)
0809 04:49:45.866889 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.866947 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 04:49:45.867194 387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45837: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
I20260809 04:49:45.916476 387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } attempt: 1
I20260809 04:49:45.917570 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.917177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38832)
0809 04:49:45.917410 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.917486 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 04:49:45.918462 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.918185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38848)
0809 04:49:45.918314 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.918396 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 04:49:45.918624 387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45837: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
I20260809 04:49:45.959722 310 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } attempt: 4
Aug 09 04:49:45 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
I20260809 04:49:45.970506 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.960379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:46161 (local address 127.0.0.1:44638)
0809 04:49:45.960629 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.960644 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:45.960739 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:45.961068 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:45.961071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:45.961243 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:45.961346 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:45.961351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.963080 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.963083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:45.964219 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:45.964232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.964841 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.964844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:45.964924 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:45.965523 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:45.965536 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:45.966804 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:45.968710 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:45.968713 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:45.968715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:45.969042 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:45.969358 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:45.969360 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:45.969362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:45.969467 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:45.969978 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:45.969981 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:45.970090 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 04:49:45.970199 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 04:49:45.970345 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260809 04:49:45.970577 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.960504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44638 (local address 127.7.171.61:46161)
0809 04:49:45.960678 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:45.960682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:45.960701 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:49:45.960844 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:45.960847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:45.960904 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:45.960998 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:45.961472 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.962948 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.964341 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:45.964726 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:45.964848 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:45.966942 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:45.966946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:45.966948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:45.966978 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:45.968579 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:45.969154 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:45.969159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:45.969163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:45.969255 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:45.969567 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:45.969570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:45.969572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:45.969821 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:45.969921 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:45.970267 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 04:49:45.970402 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:49:45.971503 310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:45.972431 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.972147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38864)
0809 04:49:45.972288 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.972356 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 04:49:45.973207 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.972962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38874)
0809 04:49:45.973093 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.973152 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 04:49:45.973348 310 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45837: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
W20260809 04:49:45.980288 393 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:45.980623 393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:45.980683 393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:45.980760 393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:45.985361 393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:45.985442 393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:45.985479 393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:45.985520 393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:45.985556 393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:45.986819 387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } attempt: 2
I20260809 04:49:45.987607 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.987327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38886)
0809 04:49:45.987469 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.987533 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 04:49:45.988350 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:45.988147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38898)
0809 04:49:45.988248 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:45.988293 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 04:49:45.988485 387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45837: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
I20260809 04:49:45.991993 393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:42369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:35105,127.7.171.61:46161,127.7.171.60:45837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab
--principal=kudu/127.7.171.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.60:45837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.171.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:45.993220 393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:45.994339 393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:46.003362 400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:46.003480 399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:46.003611 393 server_base.cc:1034] running on GCE node
W20260809 04:49:46.003520 402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:46.004456 393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:46.005626 393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:46.006795 393 hybrid_clock.cc:648] HybridClock initialized: now 1786250986006778 us; error 29 us; skew 500 ppm
Aug 09 04:49:46 dist-test-slave-1zqn krb5kdc[32692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:46.009758 393 init.cc:377] Logged in from keytab as kudu/127.7.171.60@KRBTEST.COM (short username kudu)
I20260809 04:49:46.010725 393 webserver.cc:460] Webserver started at http://127.7.171.60:35171/ using document root <none> and password file <none>
I20260809 04:49:46.011405 393 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:46.011475 393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:46.011700 393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:46.013811 393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/data/instance:
uuid: "0605bc4f152c431d9acbd2e8130171a4"
format_stamp: "Formatted at 2026-08-09 04:49:46 on dist-test-slave-1zqn"
server_key: "80078b472d4da741bf300b3c6ab1da39"
server_key_iv: "9ab77fad4a460703702cf2a38c23a152"
server_key_version: "encryptionkey@0"
I20260809 04:49:46.014402 393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/wal/instance:
uuid: "0605bc4f152c431d9acbd2e8130171a4"
format_stamp: "Formatted at 2026-08-09 04:49:46 on dist-test-slave-1zqn"
server_key: "80078b472d4da741bf300b3c6ab1da39"
server_key_iv: "9ab77fad4a460703702cf2a38c23a152"
server_key_version: "encryptionkey@0"
I20260809 04:49:46.018321 393 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 04:49:46.020913 410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:46.022104 393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:46.022230 393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/wal
uuid: "0605bc4f152c431d9acbd2e8130171a4"
format_stamp: "Formatted at 2026-08-09 04:49:46 on dist-test-slave-1zqn"
server_key: "80078b472d4da741bf300b3c6ab1da39"
server_key_iv: "9ab77fad4a460703702cf2a38c23a152"
server_key_version: "encryptionkey@0"
I20260809 04:49:46.022338 393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:46.032508 310 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } attempt: 1
I20260809 04:49:46.033393 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.033105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38902)
0809 04:49:46.033254 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:46.033315 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 04:49:46.034353 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.034110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38910)
0809 04:49:46.034235 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:46.034294 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 04:49:46.034519 310 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45837: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45837: connect: Connection refused (error 111)
I20260809 04:49:46.040114 393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:46.043006 393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:46.043212 393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:46.050509 393 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.60:45837
I20260809 04:49:46.050819 467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.60:45837 every 8 connection(s)
I20260809 04:49:46.051496 393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
I20260809 04:49:46.054468 469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:46.055181 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 393
I20260809 04:49:46.055338 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250697618759-7852-0/minicluster-data/master-2/wal/instance
I20260809 04:49:46.055686 7852 external_mini_cluster.cc:1428] Setting key aa2da16d07678d6b951a2116409bf013
I20260809 04:49:46.058962 469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:46 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
Aug 09 04:49:46 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:46.071437 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.057106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38916 (local address 127.7.171.60:45837)
0809 04:49:46.057527 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.057541 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.057576 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:49:46.058258 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.058308 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.058660 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.058944 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.059871 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.061222 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.062492 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.063135 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.063286 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.066049 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:46.066056 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:46.066071 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:46.066100 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:46.068114 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.068749 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.068757 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.068764 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.068911 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.069344 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.069347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.069348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.069745 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:46.069972 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:46.070282 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 04:49:46.070590 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":126,"threads_started":1}
I20260809 04:49:46.071516 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.061145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46578 (local address 127.7.171.62:35105)
0809 04:49:46.061283 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.061288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.061632 (+ 344us) server_negotiation.cc:407] Connection header received
0809 04:49:46.061775 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.061778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.061823 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.061891 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.062756 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.063458 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.064761 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.064963 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.065147 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.067859 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:46.067862 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:46.067864 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:46.067886 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:46.069715 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.070234 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.070239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.070241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.070321 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.070598 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.070602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.070604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.070792 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:46.070895 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:46.071209 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 04:49:46.071329 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:49:46.071564 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.061026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:35105 (local address 127.0.0.1:46578)
0809 04:49:46.061478 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:46.061535 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:46.061665 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:46.061977 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:46.061983 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:46.062480 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:46.062627 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:46.062633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.063589 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.063592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:46.064633 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:46.064646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.065194 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.065198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:46.065307 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.065837 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:46.065853 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:46.067715 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:46.069831 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:46.069840 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:46.069846 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:46.070131 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:46.070421 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:46.070424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:46.070426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:46.070508 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:46.070897 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:46.070914 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:46.071020 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 04:49:46.071153 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 04:49:46.071329 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":146,"threads_started":1}
I20260809 04:49:46.073107 469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:46 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
Aug 09 04:49:46 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:46.079644 387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } attempt: 3
I20260809 04:49:46.082293 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.073761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.7.171.61:46161)
0809 04:49:46.073884 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.073889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.074126 (+ 237us) server_negotiation.cc:407] Connection header received
0809 04:49:46.074225 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.074228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.074287 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.074378 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.074767 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.075326 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.076147 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.076356 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.076439 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.078500 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:46.078503 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:46.078505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:46.078528 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:46.080422 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.080926 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.080931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.080933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.081018 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.081306 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.081309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.081311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.081581 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:46.081707 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:46.082048 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 04:49:46.082159 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:49:46.082293 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.073660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:46161 (local address 127.0.0.1:44642)
0809 04:49:46.074041 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:46.074055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:46.074141 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:46.074423 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:46.074432 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:46.074562 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:46.074650 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:46.074654 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.075436 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.075440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:46.076008 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:46.076016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.076462 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.076469 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:46.076586 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.077075 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:46.077087 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:46.078338 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:46.080540 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:46.080544 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:46.080546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:46.080810 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:46.081110 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:46.081114 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:46.081116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:46.081208 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:46.081707 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:46.081712 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:46.081813 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 04:49:46.081990 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 04:49:46.082120 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311}
I20260809 04:49:46.083076 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.073875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46586 (local address 127.7.171.62:35105)
0809 04:49:46.073979 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.073983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.073998 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:49:46.074032 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.074035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.074084 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.074165 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.074877 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.075341 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.076152 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.076382 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.076499 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.079137 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:46.079140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:46.079142 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:46.079163 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:46.081090 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.081525 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.081529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.081531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.081619 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.081952 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.081956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.081957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.082163 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:46.082385 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:46.082786 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 04:49:46.082905 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 04:49:46.083580 469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:46 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
Aug 09 04:49:46 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
I20260809 04:49:46.089745 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.080180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38926)
0809 04:49:46.080307 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:46.080325 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:46.080408 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:46.080838 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:46.080842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:46.081030 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:46.081141 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:46.081152 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.082611 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.082614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:46.083470 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:46.083480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.084148 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.084152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:46.084235 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.084647 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:46.084658 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:46.086047 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:46.087929 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:46.087934 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:46.087937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:46.088212 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:46.088549 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:46.088553 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:46.088556 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:46.088658 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:46.089182 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:46.089187 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:46.089303 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 04:49:46.089416 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 04:49:46.089572 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 04:49:46.090430 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.080266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38926 (local address 127.7.171.60:45837)
0809 04:49:46.080607 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.080613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.080626 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:49:46.080660 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.080663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.080701 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.080779 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.081287 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.082466 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.083613 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.083960 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.084453 (+ 493us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.086172 (+ 1719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:46.086176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:46.086179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:46.086201 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:46.087827 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.088345 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.088349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.088352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.088459 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.088775 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.088778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.088781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.089042 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:46.090059 (+ 1017us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:46.090133 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 04:49:46.090249 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"spinlock_wait_cycles":6400}
Aug 09 04:49:46 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
I20260809 04:49:46.093870 387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17: Bootstrap starting.
I20260809 04:49:46.094000 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.084215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38942 (local address 127.7.171.60:45837)
0809 04:49:46.085094 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.085104 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.085115 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:49:46.085149 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.085153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.085195 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.085283 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.085811 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.086627 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.087664 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.088090 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.088180 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.090129 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:46.090132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:46.090134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:46.090155 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:46.092173 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.092650 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.092655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.092658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.092735 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.092991 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.092995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.092997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.093185 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:46.093292 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:46.093703 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 04:49:46.093835 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":791,"thread_start_us":127,"threads_started":1}
I20260809 04:49:46.094233 473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.084179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38942)
0809 04:49:46.084378 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:46.084394 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:46.084473 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:46.085393 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:46.085398 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:46.085595 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:46.085673 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:46.085680 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.086727 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.086734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:46.087552 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:46.087567 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.088179 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.088182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:46.088305 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.088839 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:46.088856 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:46.090014 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:46.092304 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:46.092307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:46.092309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:46.092544 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:46.092823 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:46.092825 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:46.092827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:46.092909 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:46.093482 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:46.093484 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:46.093551 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 04:49:46.093649 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 04:49:46.093781 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":24,"spinlock_wait_cycles":3456}
I20260809 04:49:46.094957 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.084663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.7.171.61:46161)
0809 04:49:46.084786 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.084792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.084826 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:49:46.084929 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.084932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.084975 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.085048 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.085887 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.086627 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.087578 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.087897 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.087995 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.091092 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:46.091097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:46.091101 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:46.091129 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:46.092813 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.093303 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.093309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.093311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.093392 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.093687 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.093691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.093692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.093891 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:46.094114 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:46.094709 (+ 595us) server_negotiation.cc:299] Negotiation successful
0809 04:49:46.094817 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:49:46.096756 387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:46.097585 387 log.cc:824] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:46.098503 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:46.099987 469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4: Bootstrap starting.
I20260809 04:49:46.099732 32750 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b85bd9e698c34e57bf8944fcaf97a227"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46586
I20260809 04:49:46.101676 310 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } attempt: 2
I20260809 04:49:46.102378 469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:46.103233 469 log.cc:824] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:46.105119 387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17: No bootstrap required, opened a new log
I20260809 04:49:46.105506 469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4: No bootstrap required, opened a new log
Aug 09 04:49:46 dist-test-slave-1zqn krb5kdc[32692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
I20260809 04:49:46.109421 387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } }
I20260809 04:49:46.109417 469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } }
I20260809 04:49:46.109653 469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0605bc4f152c431d9acbd2e8130171a4, State: Initialized, Role: FOLLOWER
I20260809 04:49:46.109653 387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a02fb34b4894d54bffd7dccc326ba17, State: Initialized, Role: FOLLOWER
I20260809 04:49:46.110011 469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } }
I20260809 04:49:46.110011 387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } }
I20260809 04:49:46.110881 477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } } }
I20260809 04:49:46.111202 477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:46.111410 469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:46.111419 387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:46.111464 313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.102133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45837 (local address 127.0.0.1:38944)
0809 04:49:46.102241 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:46.102256 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:46.102332 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:46.102717 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:46.102721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:46.102923 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:46.103039 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:46.103047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.104112 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.104116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:46.104984 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:46.104995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.105645 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.105648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:46.105735 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.106254 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:46.106268 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:46.107706 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:46.109754 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:46.109759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:46.109762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:46.110040 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:46.110348 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:46.110351 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:46.110354 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:46.110450 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:46.110908 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:46.110912 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:46.111033 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 04:49:46.111128 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 04:49:46.111284 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 04:49:46.112181 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.102187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38944 (local address 127.7.171.60:45837)
0809 04:49:46.102480 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.102485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.102497 (+ 12us) server_negotiation.cc:407] Connection header received
0809 04:49:46.102547 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.102552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.102594 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.102667 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.103156 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.103943 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.105152 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.105518 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.105975 (+ 457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.107867 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:46.107871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:46.107874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:46.107897 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:46.109622 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.110169 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.110172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.110174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.110254 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.110543 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.110548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.110550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.110782 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:46.111771 (+ 989us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:46.111847 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 04:49:46.111986 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
I20260809 04:49:46.113699 478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } } }
I20260809 04:49:46.113850 478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:46.115410 310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227: Bootstrap starting.
W20260809 04:49:46.117565 500 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:46.117645 500 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:46.115788 317 master_runner.cc:186] Error getting master registration for 127.7.171.62:35105: OK, instance_id { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" instance_seqno: 1786250985572611 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:49:46.119611 310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:46.119855 317 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:49:46.120597 310 log.cc:824] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:46.122731 310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227: No bootstrap required, opened a new log
W20260809 04:49:46.122985 502 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:46.123062 502 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:46.120788 393 master_runner.cc:186] Error getting master registration for 127.7.171.62:35105: OK, instance_id { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" instance_seqno: 1786250985572611 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:49:46.123931 393 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:49:46.125967 310 raft_consensus.cc:348] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } }
I20260809 04:49:46.126200 310 raft_consensus.cc:729] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b85bd9e698c34e57bf8944fcaf97a227, State: Initialized, Role: FOLLOWER
I20260809 04:49:46.126535 310 consensus_queue.cc:260] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } }
I20260809 04:49:46.127218 506 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } } }
I20260809 04:49:46.127456 506 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:46.128073 310 sys_catalog.cc:564] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:46.128141 32750 raft_consensus.cc:482] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 04:49:46.128258 32750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:46.129592 32750 raft_consensus.cc:504] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } }
I20260809 04:49:46.131362 360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b85bd9e698c34e57bf8944fcaf97a227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9a02fb34b4894d54bffd7dccc326ba17"
I20260809 04:49:46.131610 360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:46.133668 360 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b85bd9e698c34e57bf8944fcaf97a227 in term 1.
I20260809 04:49:46.134146 439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b85bd9e698c34e57bf8944fcaf97a227" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0605bc4f152c431d9acbd2e8130171a4"
I20260809 04:49:46.134373 439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:46.135663 439 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b85bd9e698c34e57bf8944fcaf97a227 in term 1.
I20260809 04:49:46.130541 32750 leader_election.cc:288] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [CANDIDATE]: Term 1 election: Requested vote from peers 9a02fb34b4894d54bffd7dccc326ba17 (127.7.171.61:46161), 0605bc4f152c431d9acbd2e8130171a4 (127.7.171.60:45837)
I20260809 04:49:46.137463 32724 leader_election.cc:302] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a02fb34b4894d54bffd7dccc326ba17, b85bd9e698c34e57bf8944fcaf97a227; no voters:
I20260809 04:49:46.142473 32708 master_runner.cc:263] No leader master found from master b85bd9e698c34e57bf8944fcaf97a227
I20260809 04:49:46.142539 32708 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:49:46.142697 506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:46.142935 506 raft_consensus.cc:686] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [term 1 LEADER]: Becoming Leader. State: Replica: b85bd9e698c34e57bf8944fcaf97a227, State: Running, Role: LEADER
I20260809 04:49:46.143304 506 consensus_queue.cc:237] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } }
I20260809 04:49:46.145215 506 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b85bd9e698c34e57bf8944fcaf97a227. Latest consensus state: current_term: 1 leader_uuid: "b85bd9e698c34e57bf8944fcaf97a227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } } }
I20260809 04:49:46.145357 506 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [sys.catalog]: This master's current role is: LEADER
W20260809 04:49:46.151218 517 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:46.153337 517 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:46.153540 528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:46.154611 528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:46.155519 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.138449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46602 (local address 127.7.171.62:35105)
0809 04:49:46.138557 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.138562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.140324 (+ 1762us) server_negotiation.cc:407] Connection header received
0809 04:49:46.140517 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.140520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.140567 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.140617 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.141659 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.142449 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.145043 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.145402 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.145484 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.150567 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:46.150571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:46.150575 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:46.150602 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:46.152345 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.153927 (+ 1582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.153931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.153933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.154025 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.154356 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.154359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.154361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.154610 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:46.154737 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:46.155151 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 04:49:46.155306 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 04:49:46.159796 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.139794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.7.171.60:45837)
0809 04:49:46.139914 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.139929 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.145619 (+ 5690us) server_negotiation.cc:407] Connection header received
0809 04:49:46.145780 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.145783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.145835 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.145893 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.151465 (+ 5572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.152427 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.153735 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.154072 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.154150 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.156011 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:46.156015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:46.156018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:46.156045 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:46.157707 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.158211 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.158217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.158219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.158318 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.158717 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.158721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.158724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.158955 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:46.159070 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:46.159467 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 04:49:46.159607 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 04:49:46.161827 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:46.139185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44662 (local address 127.7.171.61:46161)
0809 04:49:46.139291 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.139295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.143801 (+ 4506us) server_negotiation.cc:407] Connection header received
0809 04:49:46.143979 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.143983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.144030 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.144099 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.146252 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.147047 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.149443 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:46.149786 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:46.149861 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:46.155864 (+ 6003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:46.155869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:46.155871 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:46.155917 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:46.159288 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.160105 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.160109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.160110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.160204 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:46.160859 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:46.160863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:46.160865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:46.161096 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:46.161191 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:46.161549 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 04:49:46.161674 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 04:49:46.170987 439 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [term 1 FOLLOWER]: Refusing update from remote peer b85bd9e698c34e57bf8944fcaf97a227: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:46.171779 360 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [term 1 FOLLOWER]: Refusing update from remote peer b85bd9e698c34e57bf8944fcaf97a227: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:46.172606 506 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:46.173059 506 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:46.181424 478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b85bd9e698c34e57bf8944fcaf97a227. Latest consensus state: current_term: 1 leader_uuid: "b85bd9e698c34e57bf8944fcaf97a227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } } }
I20260809 04:49:46.181560 478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:46.182348 477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b85bd9e698c34e57bf8944fcaf97a227. Latest consensus state: current_term: 1 leader_uuid: "b85bd9e698c34e57bf8944fcaf97a227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } } }
I20260809 04:49:46.182505 477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:46.185822 526 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b85bd9e698c34e57bf8944fcaf97a227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } } }
I20260809 04:49:46.185974 526 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:46.186100 530 mvcc.cc:205] Tried to move back new op lower bound from 7316484039335538688 to 7316484039246909440. Current Snapshot: MvccSnapshot[applied={T|T < 7316484039335538688}]
I20260809 04:49:46.187203 478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b85bd9e698c34e57bf8944fcaf97a227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } } }
I20260809 04:49:46.187340 478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:46.189880 532 mvcc.cc:205] Tried to move back new op lower bound from 7316484039335538688 to 7316484039246909440. Current Snapshot: MvccSnapshot[applied={T|T < 7316484039335538688}]
I20260809 04:49:46.190605 506 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b85bd9e698c34e57bf8944fcaf97a227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } } }
I20260809 04:49:46.190718 506 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:46.191646 477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b85bd9e698c34e57bf8944fcaf97a227" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85bd9e698c34e57bf8944fcaf97a227" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 35105 } } peers { permanent_uuid: "9a02fb34b4894d54bffd7dccc326ba17" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 46161 } } peers { permanent_uuid: "0605bc4f152c431d9acbd2e8130171a4" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45837 } } }
I20260809 04:49:46.191757 477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:46.194254 531 mvcc.cc:205] Tried to move back new op lower bound from 7316484039335538688 to 7316484039246909440. Current Snapshot: MvccSnapshot[applied={T|T < 7316484039335538688}]
I20260809 04:49:46.196297 528 catalog_manager.cc:1347] Generated new cluster ID: 960c0192958c4beea0b6021ba67bb89f
I20260809 04:49:46.196363 528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:46.233577 528 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:46.235018 528 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:46.257352 528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b85bd9e698c34e57bf8944fcaf97a227: Generated new TSK 0
I20260809 04:49:46.258360 528 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:49:46.296286 312 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.293602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46608 (local address 127.7.171.62:35105)
0809 04:49:46.293727 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.293731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.295004 (+ 1273us) server_negotiation.cc:407] Connection header received
0809 04:49:46.295233 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.295237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.295305 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.295372 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.296127 (+ 755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46608: BlockingRecv error: recv got EOF from 127.0.0.1:46608 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 04:49:46.297961 392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.293161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.7.171.61:46161)
0809 04:49:46.293323 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.293329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.296976 (+ 3647us) server_negotiation.cc:407] Connection header received
0809 04:49:46.297176 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.297179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.297230 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.297284 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.297823 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44676: BlockingRecv error: recv got EOF from 127.0.0.1:44676 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 04:49:46.299603 474 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.294370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.7.171.60:45837)
0809 04:49:46.294503 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.294507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.298621 (+ 4114us) server_negotiation.cc:407] Connection header received
0809 04:49:46.298836 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.298839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.298893 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.298951 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.299483 (+ 532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38966: BlockingRecv error: recv got EOF from 127.0.0.1:38966 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 04:49:46.304682 392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.303032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44678 (local address 127.7.171.61:46161)
0809 04:49:46.303146 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.303150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.303625 (+ 475us) server_negotiation.cc:407] Connection header received
0809 04:49:46.303854 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.303857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.303934 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.303994 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.304568 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44678: BlockingRecv error: recv got EOF from 127.0.0.1:44678 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 04:49:46.310626 392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.308794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44690 (local address 127.7.171.61:46161)
0809 04:49:46.308918 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.308922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.309544 (+ 622us) server_negotiation.cc:407] Connection header received
0809 04:49:46.309788 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.309792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.309849 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.309906 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.310497 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44690: BlockingRecv error: recv got EOF from 127.0.0.1:44690 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 04:49:46.316980 392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.315341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.7.171.61:46161)
0809 04:49:46.315449 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.315453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.315968 (+ 515us) server_negotiation.cc:407] Connection header received
0809 04:49:46.316197 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.316201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.316251 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.316302 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.316868 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44694: BlockingRecv error: recv got EOF from 127.0.0.1:44694 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 04:49:46.325027 392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.323425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44702 (local address 127.7.171.61:46161)
0809 04:49:46.323555 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.323559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.324075 (+ 516us) server_negotiation.cc:407] Connection header received
0809 04:49:46.324285 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.324289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.324340 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.324391 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.324915 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44702: BlockingRecv error: recv got EOF from 127.0.0.1:44702 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 04:49:46.337565 392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.335725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44718 (local address 127.7.171.61:46161)
0809 04:49:46.335847 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.335852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.336559 (+ 707us) server_negotiation.cc:407] Connection header received
0809 04:49:46.336776 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.336780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.336834 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.336887 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.337439 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44718: BlockingRecv error: recv got EOF from 127.0.0.1:44718 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 04:49:46.357924 392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.356589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.7.171.61:46161)
0809 04:49:46.356749 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.356754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.356981 (+ 227us) server_negotiation.cc:407] Connection header received
0809 04:49:46.357154 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.357158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.357213 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.357298 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.357780 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44726: BlockingRecv error: recv got EOF from 127.0.0.1:44726 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 04:49:46.393733 392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.392396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44738 (local address 127.7.171.61:46161)
0809 04:49:46.392653 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.392657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.392683 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:49:46.392809 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.392812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.392860 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.392960 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.393622 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44738: BlockingRecv error: recv got EOF from 127.0.0.1:44738 (error 108)
Metrics: {"server-negotiator.queue_time_us":169}
W20260809 04:49:46.461503 392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.460189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.7.171.61:46161)
0809 04:49:46.460353 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.460358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.460492 (+ 134us) server_negotiation.cc:407] Connection header received
0809 04:49:46.460672 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.460675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.460729 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.460815 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.461370 (+ 555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44752: BlockingRecv error: recv got EOF from 127.0.0.1:44752 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 04:49:46.593039 392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.591771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44754 (local address 127.7.171.61:46161)
0809 04:49:46.591946 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.591950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.592127 (+ 177us) server_negotiation.cc:407] Connection header received
0809 04:49:46.592242 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.592245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.592296 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.592371 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.592916 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44754: BlockingRecv error: recv got EOF from 127.0.0.1:44754 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 04:49:46.852973 392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:46.851712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44760 (local address 127.7.171.61:46161)
0809 04:49:46.851880 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:46.851884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:46.851902 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:49:46.852165 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:46.852168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:46.852232 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:46.852314 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:46.852820 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44760: BlockingRecv error: recv got EOF from 127.0.0.1:44760 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 04:49:47.118993 500 catalog_manager.cc:1259] Loaded cluster ID: 960c0192958c4beea0b6021ba67bb89f
I20260809 04:49:47.119091 500 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4: loading cluster ID for follower catalog manager: success
I20260809 04:49:47.120761 500 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4: acquiring CA information for follower catalog manager: success
I20260809 04:49:47.121866 500 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0605bc4f152c431d9acbd2e8130171a4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:49:47.124215 502 catalog_manager.cc:1259] Loaded cluster ID: 960c0192958c4beea0b6021ba67bb89f
I20260809 04:49:47.124301 502 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17: loading cluster ID for follower catalog manager: success
I20260809 04:49:47.125864 502 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17: acquiring CA information for follower catalog manager: success
I20260809 04:49:47.127010 502 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9a02fb34b4894d54bffd7dccc326ba17: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:49:47.371399 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:47.367335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44768 (local address 127.7.171.61:46161)
0809 04:49:47.367624 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:47.367628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:47.367646 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:49:47.367701 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:47.367703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:47.367825 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:47.367925 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:47.368273 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:47.368821 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:47.369733 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:47.369963 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:47.370063 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:47.370393 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:47.370855 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:47.370952 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:47.371108 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 04:49:47.371164 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":110,"threads_started":1}
I20260809 04:49:47.378266 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:47.374765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38978 (local address 127.7.171.60:45837)
0809 04:49:47.375008 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:47.375013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:47.375031 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:49:47.375115 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:47.375117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:47.375235 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:47.375333 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:47.375552 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:47.376105 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:47.376908 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:47.377155 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:47.377236 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:47.377534 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:47.377955 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:47.378050 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:47.378100 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 04:49:47.378147 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
I20260809 04:49:47.385516 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:47.381406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.7.171.61:46161)
0809 04:49:47.381579 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:47.381584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:47.381691 (+ 107us) server_negotiation.cc:407] Connection header received
0809 04:49:47.381926 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:47.381930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:47.381985 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:47.382033 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:47.382409 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:47.383105 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:47.384195 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:47.384544 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:47.384659 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:47.385014 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:47.385096 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:47.385147 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:47.385332 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 04:49:47.385390 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:49:47.386096 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:47.381851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38992 (local address 127.7.171.60:45837)
0809 04:49:47.381959 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:47.381963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:47.382222 (+ 259us) server_negotiation.cc:407] Connection header received
0809 04:49:47.382332 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:47.382335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:47.382401 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:47.382469 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:47.382789 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:47.383529 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:47.384804 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:47.385113 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:47.385189 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:47.385586 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:47.385673 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:47.385721 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:47.385929 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 04:49:47.385981 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 04:49:47.386288 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:47.381383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46614 (local address 127.7.171.62:35105)
0809 04:49:47.382043 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:47.382048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:47.382062 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:49:47.382159 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:47.382162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:47.382336 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:47.382456 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:47.382695 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:47.383520 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:47.384612 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:47.384845 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:47.384959 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:47.385258 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:47.385619 (+ 361us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:47.386023 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:47.386059 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 04:49:47.386118 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":90,"threads_started":1}
I20260809 04:49:47.395243 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 32708
I20260809 04:49:47.405328 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 317
I20260809 04:49:47.414425 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 393
2026-08-09T04:49:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3300 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:47 dist-test-slave-1zqn krb5kdc[628](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:47 dist-test-slave-1zqn krb5kdc[628](info): set up 2 sockets
Aug 09 04:49:47 dist-test-slave-1zqn krb5kdc[628](info): commencing operation
krb5kdc: starting...
W20260809 04:49:48.541152 7852 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/test-user.keytab.
Aug 09 04:49:48 dist-test-slave-1zqn krb5kdc[628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.171.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
Entry for principal kudu/127.7.171.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab.
I20260809 04:49:48.683889 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46667
--webserver_interface=127.7.171.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:46667,127.7.171.61:44703,127.7.171.60:45319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:48.800679 642 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:48.801046 642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:48.801112 642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:48.801189 642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:48.805545 642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:48.805642 642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:48.805684 642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:48.805721 642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:48.805755 642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:48.811336 642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:46667,127.7.171.61:44703,127.7.171.60:45319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.62.keytab
--principal=kudu/127.7.171.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.62:46667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.171.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:48.812542 642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:48.813640 642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:48.823487 649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:48.823513 651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:48.823727 642 server_base.cc:1034] running on GCE node
W20260809 04:49:48.823475 648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:48.824609 642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:48.826066 642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:48.827291 642 hybrid_clock.cc:648] HybridClock initialized: now 1786250988827251 us; error 57 us; skew 500 ppm
Aug 09 04:49:48 dist-test-slave-1zqn krb5kdc[628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250988, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:48.830466 642 init.cc:377] Logged in from keytab as kudu/127.7.171.62@KRBTEST.COM (short username kudu)
I20260809 04:49:48.831501 642 webserver.cc:460] Webserver started at http://127.7.171.62:37183/ using document root <none> and password file <none>
I20260809 04:49:48.832172 642 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:48.832269 642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:48.832517 642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:48.834811 642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/data/instance:
uuid: "86efa4479a3f429f82c24389f99db4f4"
format_stamp: "Formatted at 2026-08-09 04:49:48 on dist-test-slave-1zqn"
server_key: "4ea31ed5983dca07b0eb3e19c4312489"
server_key_iv: "cea381119b96b6381dac8e6dac361c0c"
server_key_version: "encryptionkey@0"
I20260809 04:49:48.835414 642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/wal/instance:
uuid: "86efa4479a3f429f82c24389f99db4f4"
format_stamp: "Formatted at 2026-08-09 04:49:48 on dist-test-slave-1zqn"
server_key: "4ea31ed5983dca07b0eb3e19c4312489"
server_key_iv: "cea381119b96b6381dac8e6dac361c0c"
server_key_version: "encryptionkey@0"
I20260809 04:49:48.839524 642 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:49:48.842127 659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:48.843535 642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:49:48.843708 642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/wal
uuid: "86efa4479a3f429f82c24389f99db4f4"
format_stamp: "Formatted at 2026-08-09 04:49:48 on dist-test-slave-1zqn"
server_key: "4ea31ed5983dca07b0eb3e19c4312489"
server_key_iv: "cea381119b96b6381dac8e6dac361c0c"
server_key_version: "encryptionkey@0"
I20260809 04:49:48.843842 642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:48.893558 642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:48.896968 642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:48.897193 642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:48.905404 642 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.62:46667
I20260809 04:49:48.905782 712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.62:46667 every 8 connection(s)
I20260809 04:49:48.906538 642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:48.909600 713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:48.912353 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 642
I20260809 04:49:48.912536 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-0/wal/instance
I20260809 04:49:48.912856 7852 external_mini_cluster.cc:1428] Setting key 648934ffb217e02d9ac11433ee1b0ea3
I20260809 04:49:48.913836 713 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.171.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.61@KRBTEST.COM" created.
Aug 09 04:49:48 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250988, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:48.933558 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:48.916494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40528 (local address 127.7.171.62:46667)
0809 04:49:48.917134 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:48.917153 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:48.917330 (+ 177us) server_negotiation.cc:407] Connection header received
0809 04:49:48.919035 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:48.919039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:48.919279 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:48.919442 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:48.920805 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:48.921776 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:48.923426 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:48.923841 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:48.923958 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:48.928159 (+ 4201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:48.928166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:48.928178 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:48.928201 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:48.930020 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:48.930573 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:48.930578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:48.930579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:48.930690 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:48.931033 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:48.931037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:48.931039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:48.931446 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:48.931610 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:48.931998 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 04:49:48.932342 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":492,"thread_start_us":165,"threads_started":1}
I20260809 04:49:48.933558 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:48.916494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:46667 (local address 127.0.0.1:40528)
0809 04:49:48.917134 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:48.917199 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:48.918693 (+ 1494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:48.919526 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:48.919536 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:48.920155 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:48.920627 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:48.920652 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:48.921920 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:48.921927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:48.923305 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:48.923318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:48.924001 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:48.924006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:48.924183 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:48.924790 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:48.924816 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:48.927896 (+ 3080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:48.930160 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:48.930171 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:48.930182 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:48.930431 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:48.930796 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:48.930800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:48.930802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:48.930924 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:48.931578 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:48.931585 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:48.931741 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 04:49:48.931916 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 04:49:48.932398 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":117,"spinlock_wait_cycles":58112,"thread_start_us":164,"threads_started":1}
I20260809 04:49:48.936379 713 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:48.937279 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:48.936954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:44703 (local address 127.0.0.1:60718)
0809 04:49:48.937125 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:48.937203 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:44703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260809 04:49:48.937570 662 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.171.61:44703: connect: Connection refused (error 111)
I20260809 04:49:48.938449 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:48.938220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:44703 (local address 127.0.0.1:60734)
0809 04:49:48.938341 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:48.938394 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:44703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 04:49:48.938622 713 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.61:44703: Network error: Client connection negotiation failed: client connection to 127.7.171.61:44703: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab.
Entry for principal kudu/127.7.171.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab.
I20260809 04:49:48.945188 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.61:44703
--webserver_interface=127.7.171.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab
--principal=kudu/127.7.171.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:46667,127.7.171.61:44703,127.7.171.60:45319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:49:48.987893 713 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } attempt: 1
I20260809 04:49:48.989090 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:48.988621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:44703 (local address 127.0.0.1:60748)
0809 04:49:48.988847 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:48.988958 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:44703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 04:49:48.990063 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:48.989738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:44703 (local address 127.0.0.1:60750)
0809 04:49:48.989889 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:48.989964 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:44703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 04:49:48.990259 713 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.61:44703: Network error: Client connection negotiation failed: client connection to 127.7.171.61:44703: connect: Connection refused (error 111)
I20260809 04:49:49.058477 713 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } attempt: 2
I20260809 04:49:49.059523 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.059124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:44703 (local address 127.0.0.1:60752)
0809 04:49:49.059320 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.059418 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:44703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 04:49:49.060413 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.060153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:44703 (local address 127.0.0.1:60760)
0809 04:49:49.060274 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.060337 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.61:44703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 04:49:49.060590 713 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.61:44703: Network error: Client connection negotiation failed: client connection to 127.7.171.61:44703: connect: Connection refused (error 111)
W20260809 04:49:49.066174 720 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:49.066484 720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:49.066560 720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:49.066649 720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:49.071215 720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:49.071285 720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:49.071309 720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:49.071353 720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:49.071398 720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:49.077009 720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:46667,127.7.171.61:44703,127.7.171.60:45319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.61.keytab
--principal=kudu/127.7.171.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.61:44703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.171.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:49.078157 720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:49.079279 720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:49.088343 729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:49.088356 726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:49.088356 727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:49.088533 720 server_base.cc:1034] running on GCE node
I20260809 04:49:49.089268 720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:49.090481 720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:49.091674 720 hybrid_clock.cc:648] HybridClock initialized: now 1786250989091646 us; error 41 us; skew 500 ppm
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:49.094745 720 init.cc:377] Logged in from keytab as kudu/127.7.171.61@KRBTEST.COM (short username kudu)
I20260809 04:49:49.095718 720 webserver.cc:460] Webserver started at http://127.7.171.61:41349/ using document root <none> and password file <none>
I20260809 04:49:49.096587 720 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:49.096669 720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:49.097077 720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:49.099334 720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/data/instance:
uuid: "e929284c181946369435679bf0db5b4a"
format_stamp: "Formatted at 2026-08-09 04:49:49 on dist-test-slave-1zqn"
server_key: "3ab71295186450ce6e0aa0610e4ab2ac"
server_key_iv: "f1268d35ba1b45e3e69ea837dda6b891"
server_key_version: "encryptionkey@0"
I20260809 04:49:49.099951 720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/wal/instance:
uuid: "e929284c181946369435679bf0db5b4a"
format_stamp: "Formatted at 2026-08-09 04:49:49 on dist-test-slave-1zqn"
server_key: "3ab71295186450ce6e0aa0610e4ab2ac"
server_key_iv: "f1268d35ba1b45e3e69ea837dda6b891"
server_key_version: "encryptionkey@0"
I20260809 04:49:49.104012 720 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:49:49.106575 737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:49.107880 720 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:49:49.108114 720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/wal
uuid: "e929284c181946369435679bf0db5b4a"
format_stamp: "Formatted at 2026-08-09 04:49:49 on dist-test-slave-1zqn"
server_key: "3ab71295186450ce6e0aa0610e4ab2ac"
server_key_iv: "f1268d35ba1b45e3e69ea837dda6b891"
server_key_version: "encryptionkey@0"
I20260809 04:49:49.108263 720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:49.126788 720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:49.130477 720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:49.130642 720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:49.137753 720 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.61:44703
I20260809 04:49:49.138108 793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.61:44703 every 8 connection(s)
I20260809 04:49:49.138826 720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/data/info.pb
I20260809 04:49:49.141994 794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:49.143414 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 720
I20260809 04:49:49.143574 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-1/wal/instance
I20260809 04:49:49.143947 7852 external_mini_cluster.cc:1428] Setting key 109d38bf324e7ae444208a4b24609886
I20260809 04:49:49.147433 794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:49.151785 713 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } attempt: 3
WARNING: no policy specified for kudu/127.7.171.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.171.60@KRBTEST.COM" created.
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250988, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:49.163570 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.152292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:44703 (local address 127.0.0.1:60768)
0809 04:49:49.152526 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.152546 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:49.152629 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:49.153296 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:49.153299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:49.153511 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:49.153644 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.153652 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.155290 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.155303 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.156466 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.156475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.157110 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.157114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.157228 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.157726 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:49.157746 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:49.159315 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:49.161512 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.161517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.161519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.161810 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.162209 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.162213 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.162216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.162315 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.162968 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:49.162971 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:49.163077 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 04:49:49.163197 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 04:49:49.163354 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":4}
I20260809 04:49:49.165122 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.149592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.7.171.62:46667)
0809 04:49:49.149748 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.149753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.150707 (+ 954us) server_negotiation.cc:407] Connection header received
0809 04:49:49.151810 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.151814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.151886 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.152000 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.153654 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.154399 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.155813 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.156361 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.156495 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.160277 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.160284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.160289 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.160321 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.163054 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.163603 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.163608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.163613 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.163722 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.164092 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.164095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.164097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.164316 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.164417 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.164863 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.164994 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:49:49.165378 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.150021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:46667 (local address 127.0.0.1:40540)
0809 04:49:49.150540 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.150597 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:49.151573 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:49.152327 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:49.152340 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:49.152838 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:49.153497 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.153513 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.154532 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.154535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.155664 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.155679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.156572 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.156577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.156828 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.157678 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:49.157706 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:49.160122 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:49.163181 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.163192 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.163198 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.163492 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.163857 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.163863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.163865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.163972 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.164450 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:49.164459 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:49.164605 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 04:49:49.164783 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 04:49:49.164987 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":150,"threads_started":1}
I20260809 04:49:49.165385 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.152398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60768 (local address 127.7.171.61:44703)
0809 04:49:49.152851 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.152861 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.152880 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:49:49.152917 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.152921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.153139 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.153311 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.153774 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.154866 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.156590 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.156987 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.157107 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.159475 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.159482 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.159497 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.159530 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.161387 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.161923 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.161934 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.161940 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.162080 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.162426 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.162428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.162429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.162844 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.162995 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.163928 (+ 933us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.164302 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":161,"threads_started":1}
I20260809 04:49:49.167382 794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:49.167630 713 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:49.168299 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.168030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60012)
0809 04:49:49.168191 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.168242 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 04:49:49.169205 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.168904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60018)
0809 04:49:49.169078 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.169142 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260809 04:49:49.169353 713 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45319: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
I20260809 04:49:49.176424 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.167991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:44703 (local address 127.0.0.1:60782)
0809 04:49:49.168189 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.168204 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:49.168287 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:49.168725 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:49.168731 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:49.168915 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:49.169042 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.169050 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.170020 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.170023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.170590 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.170598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.171063 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.171082 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.171193 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.171592 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:49.171603 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:49.172953 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:49.174845 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.174848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.174849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.175036 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.175355 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.175357 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.175358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.175434 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.175852 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:49.175854 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:49.175931 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 04:49:49.176096 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 04:49:49.176230 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260809 04:49:49.176487 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.168080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60782 (local address 127.7.171.61:44703)
0809 04:49:49.168487 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.168490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.168501 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:49:49.168545 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.168548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.168592 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.168677 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.169181 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.169917 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.170695 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.170950 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.171059 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.173091 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.173094 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.173096 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.173116 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.174739 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.175167 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.175170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.175171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.175267 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.175524 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.175526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.175527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.175763 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.175842 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.176129 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.176323 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":336}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.171.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab.
Entry for principal kudu/127.7.171.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab.
I20260809 04:49:49.177673 7852 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
/tmp/dist-test-taskfTSqH0/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.60:45319
--webserver_interface=127.7.171.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab
--principal=kudu/127.7.171.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.171.20:36817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.171.62:46667,127.7.171.61:44703,127.7.171.60:45319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:49:49.179626 794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:49.180522 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.180259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60028)
0809 04:49:49.180371 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.180448 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 04:49:49.181285 738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
I20260809 04:49:49.182191 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.181975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60042)
0809 04:49:49.182096 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.182142 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 04:49:49.182427 794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45319: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
I20260809 04:49:49.200510 713 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } attempt: 1
I20260809 04:49:49.201439 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.201110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60056)
0809 04:49:49.201269 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.201345 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 04:49:49.202337 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.202097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60068)
0809 04:49:49.202222 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.202278 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:49:49.202487 713 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45319: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
I20260809 04:49:49.231696 794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } attempt: 1
I20260809 04:49:49.232898 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.232592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60074)
0809 04:49:49.232747 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.232812 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 04:49:49.233732 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.233445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60080)
0809 04:49:49.233585 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.233655 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 04:49:49.233858 794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45319: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
I20260809 04:49:49.277668 713 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } attempt: 2
I20260809 04:49:49.278714 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.278402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60090)
0809 04:49:49.278553 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.278623 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 04:49:49.279435 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.279212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60106)
0809 04:49:49.279329 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.279378 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 04:49:49.279575 713 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45319: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
W20260809 04:49:49.295269 801 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:49.295606 801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:49.295665 801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:49.295746 801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:49.299808 801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:49.299892 801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:49.299932 801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:49.299984 801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:49.300016 801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:49.302008 794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } attempt: 2
I20260809 04:49:49.302829 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.302547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60122)
0809 04:49:49.302688 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.302753 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 04:49:49.303603 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.303382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60136)
0809 04:49:49.303501 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.303551 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 04:49:49.303725 794 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.171.60:45319: Network error: Client connection negotiation failed: client connection to 127.7.171.60:45319: connect: Connection refused (error 111)
I20260809 04:49:49.305644 801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.171.20:36817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.171.62:46667,127.7.171.61:44703,127.7.171.60:45319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/krb5kdc/kudu_127.7.171.60.keytab
--principal=kudu/127.7.171.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.171.60:45319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.171.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 03:57:59 UTC on ccycloud.quasar-dlmoqf.root.comops.site
build id 4575792
I20260809 04:49:49.306697 801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:49.307710 801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:49.316251 807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:49.316262 810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:49.316491 801 server_base.cc:1034] running on GCE node
W20260809 04:49:49.316259 808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:49.317231 801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:49.318342 801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:49.319538 801 hybrid_clock.cc:648] HybridClock initialized: now 1786250989319516 us; error 35 us; skew 500 ppm
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:49.322471 801 init.cc:377] Logged in from keytab as kudu/127.7.171.60@KRBTEST.COM (short username kudu)
I20260809 04:49:49.323304 801 webserver.cc:460] Webserver started at http://127.7.171.60:33275/ using document root <none> and password file <none>
I20260809 04:49:49.323823 801 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:49.323900 801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:49.324137 801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:49.326144 801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/data/instance:
uuid: "930dcee525574bcabd0b78d85116b387"
format_stamp: "Formatted at 2026-08-09 04:49:49 on dist-test-slave-1zqn"
server_key: "ce0cdf8b1c1b970a2e26f6768dfc2348"
server_key_iv: "5896b9dded3abc5900c43115a6d91a3a"
server_key_version: "encryptionkey@0"
I20260809 04:49:49.326675 801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/wal/instance:
uuid: "930dcee525574bcabd0b78d85116b387"
format_stamp: "Formatted at 2026-08-09 04:49:49 on dist-test-slave-1zqn"
server_key: "ce0cdf8b1c1b970a2e26f6768dfc2348"
server_key_iv: "5896b9dded3abc5900c43115a6d91a3a"
server_key_version: "encryptionkey@0"
I20260809 04:49:49.330452 801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260809 04:49:49.332901 817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:49.334193 801 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 04:49:49.334311 801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/data,/tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/wal
uuid: "930dcee525574bcabd0b78d85116b387"
format_stamp: "Formatted at 2026-08-09 04:49:49 on dist-test-slave-1zqn"
server_key: "ce0cdf8b1c1b970a2e26f6768dfc2348"
server_key_iv: "5896b9dded3abc5900c43115a6d91a3a"
server_key_version: "encryptionkey@0"
I20260809 04:49:49.334398 801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:49.343752 801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:49.347028 801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:49.347237 801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:49.354260 801 rpc_server.cc:307] RPC server started. Bound to: 127.7.171.60:45319
I20260809 04:49:49.354537 872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.171.60:45319 every 8 connection(s)
I20260809 04:49:49.355299 801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/data/info.pb
I20260809 04:49:49.355763 7852 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu as pid 801
I20260809 04:49:49.355916 7852 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfTSqH0/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250697618759-7852-0/minicluster-data/master-2/wal/instance
I20260809 04:49:49.356355 7852 external_mini_cluster.cc:1428] Setting key e426f5a13631bd20040cdc5ca7d60962
I20260809 04:49:49.358737 873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:49.362483 873 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
I20260809 04:49:49.372543 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.357784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60138 (local address 127.7.171.60:45319)
0809 04:49:49.358322 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.358339 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.358378 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:49:49.359283 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.359302 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.359677 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.360088 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.360986 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.362013 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.362922 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.363590 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.363755 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.366738 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.366744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.366753 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.366782 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.368867 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.369354 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.369366 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.369370 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.369492 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.369820 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.369822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.369823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.370386 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.370607 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.371160 (+ 553us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.371467 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":155,"threads_started":1}
I20260809 04:49:49.373904 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.364525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.7.171.62:46667)
0809 04:49:49.364693 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.364700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.365011 (+ 311us) server_negotiation.cc:407] Connection header received
0809 04:49:49.365177 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.365180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.365233 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.365315 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.365976 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.366554 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.367435 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.367673 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.367782 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.370048 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.370051 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.370054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.370078 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.371671 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.372307 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.372310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.372312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.372406 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.372851 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.372856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.372858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.373106 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.373261 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.373603 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.373730 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:49:49.374050 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.364490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.62:46667 (local address 127.0.0.1:40544)
0809 04:49:49.364894 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.364933 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:49.365071 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:49.365365 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:49.365370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:49.365729 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:49.365851 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.365857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.366673 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.366679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.367316 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.367324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.367785 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.367788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.367905 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.368398 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:49.368409 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:49.369912 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:49.371791 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.371799 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.371803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.372200 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.372521 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.372524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.372525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.372671 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.373260 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:49.373268 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:49.373387 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 04:49:49.373540 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 04:49:49.373731 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":160,"threads_started":1}
I20260809 04:49:49.375173 873 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:49.378741 713 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } attempt: 3
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.62@KRBTEST.COM
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250988, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.62@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
I20260809 04:49:49.384668 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.375788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60790 (local address 127.7.171.61:44703)
0809 04:49:49.375941 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.375948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.375990 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:49:49.376152 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.376159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.376218 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.376309 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.376750 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.377283 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.378109 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.378370 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.378472 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.381010 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.381014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.381017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.381053 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.382624 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.383206 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.383210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.383212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.383304 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.383633 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.383638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.383642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.383882 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.383961 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.384373 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.384503 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:49:49.385155 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.375647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.61:44703 (local address 127.0.0.1:60790)
0809 04:49:49.375937 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.375949 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:49.376018 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:49.376325 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:49.376327 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:49.376493 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:49.376620 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.376627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.377395 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.377398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.377981 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.377993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.378485 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.378488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.378569 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.379341 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:49.379358 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:49.380853 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:49.382719 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.382726 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.382731 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.383013 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.383427 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.383430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.383432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.383522 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.384049 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:49.384073 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:49.384174 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 04:49:49.384817 (+ 643us) client_negotiation.cc:240] Negotiation successful
0809 04:49:49.384963 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215}
I20260809 04:49:49.386096 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.375121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.7.171.62:46667)
0809 04:49:49.375244 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.375247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.375441 (+ 194us) server_negotiation.cc:407] Connection header received
0809 04:49:49.375598 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.375601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.375666 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.375752 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.376537 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.377145 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.377970 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.378254 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.378336 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.381749 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.381753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.381756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.381783 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.383745 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.384283 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.384287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.384289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.384381 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.384691 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.384694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.384696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.385028 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.385275 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.385424 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.385551 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:49:49.386143 873 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:49.388331 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.379168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60146)
0809 04:49:49.379294 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.379308 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:49.379389 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:49.379675 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:49.379680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:49.379856 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:49.379972 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.379981 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.380795 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.380800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.381517 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.381528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.382652 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.382659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.382756 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.383205 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:49.383221 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:49.384823 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:49.386743 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.386747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.386749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.387001 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.387311 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.387314 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.387328 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.387426 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.387898 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:49.387900 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:49.387984 (+ 84us) client_negotiation.cc:769] Sending connection context
0809 04:49:49.388110 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 04:49:49.388235 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 04:49:49.388824 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.379232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60146 (local address 127.7.171.60:45319)
0809 04:49:49.379347 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.379351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.379367 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:49:49.379491 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.379498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.379547 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.379641 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.380140 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.380691 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.381664 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.382513 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.382881 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.384967 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.384970 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.384972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.384994 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.386623 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.387126 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.387132 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.387137 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.387214 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.387537 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.387540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.387542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.387789 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.388510 (+ 721us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.388589 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.388671 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.60@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
I20260809 04:49:49.392488 713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4: Bootstrap starting.
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.171.61@KRBTEST.COM
I20260809 04:49:49.394872 794 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } attempt: 3
I20260809 04:49:49.395383 713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:49.396382 713 log.cc:824] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:49.396478 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.386585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60162)
0809 04:49:49.387013 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.387027 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:49.387109 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:49.387460 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:49.387465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:49.387636 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:49.387770 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.387777 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.389692 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.389696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.390198 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.390206 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.390692 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.390696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.390822 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.391283 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:49.391301 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:49.392599 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:49.394903 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.394906 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.394908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.395165 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.395472 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.395474 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.395476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.395571 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.395983 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:49.395986 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:49.396095 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 04:49:49.396195 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 04:49:49.396325 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":15}
I20260809 04:49:49.396929 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.386748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60162 (local address 127.7.171.60:45319)
0809 04:49:49.387103 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.387117 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.387137 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:49:49.387180 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.387183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.387228 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.387328 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.387910 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.389567 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.390317 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.390583 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.390696 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.392724 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.392728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.392731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.392755 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.394545 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.395280 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.395286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.395288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.395371 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.395678 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.395682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.395684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.395868 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.395952 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.396646 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.396765 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260809 04:49:49.398522 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.387499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.7.171.61:44703)
0809 04:49:49.387620 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.387623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.387636 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:49:49.387735 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.387737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.387777 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.387858 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.388768 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.389419 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.390390 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.390669 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.390756 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.393822 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.393827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.393830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.393864 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.395842 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.396428 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.396432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.396434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.396814 (+ 380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.397287 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.397290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.397292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.397545 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.397770 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.398107 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.398235 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:49:49.400123 873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387: Bootstrap starting.
I20260809 04:49:49.400581 713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4: No bootstrap required, opened a new log
Aug 09 04:49:49 dist-test-slave-1zqn krb5kdc[628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.171.61@KRBTEST.COM for kudu/127.7.171.60@KRBTEST.COM
I20260809 04:49:49.401990 7852 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:49.403463 873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:49.403393 686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "86efa4479a3f429f82c24389f99db4f4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40550
I20260809 04:49:49.404461 873 log.cc:824] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:49.405143 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.395302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.171.60:45319 (local address 127.0.0.1:60166)
0809 04:49:49.395488 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:49.395506 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:49.395593 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:49.396240 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:49.396246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:49.396430 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:49.396546 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.396556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.397759 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.397762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.398503 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:49.398516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.399218 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.399222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:49.399299 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.400210 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:49.400224 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:49.401590 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:49.403523 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.403526 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.403528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.403789 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.404160 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:49.404163 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:49.404165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:49.404257 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:49.404711 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:49.404714 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:49.404823 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 04:49:49.404914 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 04:49:49.405035 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260809 04:49:49.405593 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.395799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60166 (local address 127.7.171.60:45319)
0809 04:49:49.395954 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.395959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.395972 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:49:49.396006 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.396009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.396086 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.396796 (+ 710us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.396915 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.397640 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.398673 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.399082 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.399503 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.401736 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.401739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.401742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.401764 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.403389 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.403909 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.403912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.403914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.404007 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.404376 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.404379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.404381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.404593 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.405328 (+ 735us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.405406 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.405489 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:49:49.405421 713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } }
I20260809 04:49:49.405681 713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86efa4479a3f429f82c24389f99db4f4, State: Initialized, Role: FOLLOWER
I20260809 04:49:49.406061 713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } }
I20260809 04:49:49.406716 881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } } }
I20260809 04:49:49.407014 881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:49.407523 713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:49.407555 873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387: No bootstrap required, opened a new log
I20260809 04:49:49.407593 686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 04:49:49.407717 686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:49.408703 794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a: Bootstrap starting.
I20260809 04:49:49.408984 686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } }
I20260809 04:49:49.409884 686 leader_election.cc:288] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [CANDIDATE]: Term 1 election: Requested vote from peers e929284c181946369435679bf0db5b4a (127.7.171.61:44703), 930dcee525574bcabd0b78d85116b387 (127.7.171.60:45319)
I20260809 04:49:49.411803 873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } }
I20260809 04:49:49.412098 873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 930dcee525574bcabd0b78d85116b387, State: Initialized, Role: FOLLOWER
I20260809 04:49:49.412431 873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } }
I20260809 04:49:49.413791 873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:49:49.413810 893 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:49.413890 893 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:49.413842 846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "86efa4479a3f429f82c24389f99db4f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "930dcee525574bcabd0b78d85116b387"
I20260809 04:49:49.414062 846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:49.415613 846 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86efa4479a3f429f82c24389f99db4f4 in term 1.
I20260809 04:49:49.416249 660 leader_election.cc:302] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86efa4479a3f429f82c24389f99db4f4, 930dcee525574bcabd0b78d85116b387; no voters:
I20260809 04:49:49.416682 881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:49.416991 881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [term 1 LEADER]: Becoming Leader. State: Replica: 86efa4479a3f429f82c24389f99db4f4, State: Running, Role: LEADER
I20260809 04:49:49.417378 881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } }
I20260809 04:49:49.417853 794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:49.419381 881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86efa4479a3f429f82c24389f99db4f4. Latest consensus state: current_term: 1 leader_uuid: "86efa4479a3f429f82c24389f99db4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } } }
I20260809 04:49:49.419595 881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:49.419677 794 log.cc:824] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:49.415745 642 master_runner.cc:186] Error getting master registration for 127.7.171.61:44703: OK, instance_id { permanent_uuid: "e929284c181946369435679bf0db5b4a" instance_seqno: 1786250989135601 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:49:49.419901 642 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 04:49:49.419299 912 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:49.420202 912 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:49.420271 916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:49.419541 901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } } }
I20260809 04:49:49.421476 916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:49.421847 901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:49.425638 794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a: No bootstrap required, opened a new log
I20260809 04:49:49.424196 801 master_runner.cc:186] Error getting master registration for 127.7.171.61:44703: OK, instance_id { permanent_uuid: "e929284c181946369435679bf0db5b4a" instance_seqno: 1786250989135601 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:49:49.427698 801 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:49:49.431610 794 raft_consensus.cc:348] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } }
I20260809 04:49:49.431893 794 raft_consensus.cc:729] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e929284c181946369435679bf0db5b4a, State: Initialized, Role: FOLLOWER
I20260809 04:49:49.432283 794 consensus_queue.cc:260] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } }
I20260809 04:49:49.432636 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.421503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40554 (local address 127.7.171.62:46667)
0809 04:49:49.421628 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.421633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.422406 (+ 773us) server_negotiation.cc:407] Connection header received
0809 04:49:49.422596 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.422599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.422652 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.422704 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.423774 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.424591 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.425957 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.426311 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.426389 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.428557 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.428561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.428564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.428593 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.430477 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.431003 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.431007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.431009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.431113 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.431433 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.431436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.431438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.431665 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.431769 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.432349 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.432476 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:49:49.433877 794 sys_catalog.cc:564] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:49.434059 765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "86efa4479a3f429f82c24389f99db4f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e929284c181946369435679bf0db5b4a"
I20260809 04:49:49.434315 765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:49.435945 765 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86efa4479a3f429f82c24389f99db4f4 in term 1.
I20260809 04:49:49.443029 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.422655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.7.171.61:44703)
0809 04:49:49.422775 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.422780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.433420 (+ 10640us) server_negotiation.cc:407] Connection header received
0809 04:49:49.433602 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.433606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.433661 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.433713 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.434808 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.435604 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.436837 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.437202 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.437277 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.439167 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.439171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.439174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.439200 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.441236 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.441731 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.441735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.441737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.441818 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.442113 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.442117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.442119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.442318 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.442416 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.442757 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.442873 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:49:49.444371 923 sys_catalog.cc:455] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } } }
I20260809 04:49:49.444614 923 sys_catalog.cc:458] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:49.445449 846 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [term 1 FOLLOWER]: Refusing update from remote peer 86efa4479a3f429f82c24389f99db4f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:49.446352 765 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [term 1 FOLLOWER]: Refusing update from remote peer 86efa4479a3f429f82c24389f99db4f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:49.447381 913 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:49.447608 881 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:49.450090 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:49.428338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.7.171.60:45319)
0809 04:49:49.428470 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.428475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.436816 (+ 8341us) server_negotiation.cc:407] Connection header received
0809 04:49:49.437026 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.437029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.437091 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.437151 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.438250 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.438961 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.440125 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:49.440464 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:49.440550 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:49.445317 (+ 4767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:49.445321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:49.445324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:49.445355 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:49.447294 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.448665 (+ 1371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.448669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.448671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.448768 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:49.449099 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:49.449103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:49.449105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:49.449324 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:49.449435 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:49.449790 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 04:49:49.449917 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:49:49.455739 901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86efa4479a3f429f82c24389f99db4f4. Latest consensus state: current_term: 1 leader_uuid: "86efa4479a3f429f82c24389f99db4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } } }
I20260809 04:49:49.455909 901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:49.458699 913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86efa4479a3f429f82c24389f99db4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } } }
I20260809 04:49:49.458839 913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:49.458981 913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86efa4479a3f429f82c24389f99db4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } } }
I20260809 04:49:49.459085 913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:49.460686 901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86efa4479a3f429f82c24389f99db4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } } }
I20260809 04:49:49.460800 901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:49.458473 923 sys_catalog.cc:455] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86efa4479a3f429f82c24389f99db4f4. Latest consensus state: current_term: 1 leader_uuid: "86efa4479a3f429f82c24389f99db4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } } }
I20260809 04:49:49.461251 923 sys_catalog.cc:458] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:49.461359 923 sys_catalog.cc:455] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86efa4479a3f429f82c24389f99db4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86efa4479a3f429f82c24389f99db4f4" member_type: VOTER last_known_addr { host: "127.7.171.62" port: 46667 } } peers { permanent_uuid: "e929284c181946369435679bf0db5b4a" member_type: VOTER last_known_addr { host: "127.7.171.61" port: 44703 } } peers { permanent_uuid: "930dcee525574bcabd0b78d85116b387" member_type: VOTER last_known_addr { host: "127.7.171.60" port: 45319 } } }
I20260809 04:49:49.461472 923 sys_catalog.cc:458] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:49.462332 920 mvcc.cc:205] Tried to move back new op lower bound from 7316484052758421504 to 7316484052657909760. Current Snapshot: MvccSnapshot[applied={T|T < 7316484052758421504}]
I20260809 04:49:49.464382 916 catalog_manager.cc:1347] Generated new cluster ID: 79d5b6329d0b40e6ace08d4ae90b27b3
I20260809 04:49:49.464457 916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:49.468211 938 mvcc.cc:205] Tried to move back new op lower bound from 7316484052758421504 to 7316484052657909760. Current Snapshot: MvccSnapshot[applied={T|T < 7316484052758421504}]
I20260809 04:49:49.473500 935 catalog_manager.cc:1259] Loaded cluster ID: 79d5b6329d0b40e6ace08d4ae90b27b3
I20260809 04:49:49.473660 935 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a: loading cluster ID for follower catalog manager: success
W20260809 04:49:49.474200 935 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260809 04:49:49.474344 935 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:49.496460 916 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:49.497828 916 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:49.512187 916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86efa4479a3f429f82c24389f99db4f4: Generated new TSK 0
I20260809 04:49:49.515383 916 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:49:49.565137 716 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.561140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40564 (local address 127.7.171.62:46667)
0809 04:49:49.561288 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.561292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.563482 (+ 2190us) server_negotiation.cc:407] Connection header received
0809 04:49:49.563872 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.563875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.563926 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.564009 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.564988 (+ 979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40564: BlockingRecv error: recv got EOF from 127.0.0.1:40564 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 04:49:49.566023 876 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.560611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.7.171.60:45319)
0809 04:49:49.560801 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.560806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.563435 (+ 2629us) server_negotiation.cc:407] Connection header received
0809 04:49:49.563833 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.563839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.563897 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.563960 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.565905 (+ 1945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60176: BlockingRecv error: recv got EOF from 127.0.0.1:60176 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260809 04:49:49.567319 799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.562419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60818 (local address 127.7.171.61:44703)
0809 04:49:49.562571 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.562576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.565678 (+ 3102us) server_negotiation.cc:407] Connection header received
0809 04:49:49.566558 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.566561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.566616 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.566704 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.567205 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60818: BlockingRecv error: recv got EOF from 127.0.0.1:60818 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 04:49:49.574882 799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.573085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60832 (local address 127.7.171.61:44703)
0809 04:49:49.573207 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.573211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.573855 (+ 644us) server_negotiation.cc:407] Connection header received
0809 04:49:49.574087 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.574091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.574191 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.574256 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.574778 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60832: BlockingRecv error: recv got EOF from 127.0.0.1:60832 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 04:49:49.579988 799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.578478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60848 (local address 127.7.171.61:44703)
0809 04:49:49.578575 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.578579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.579124 (+ 545us) server_negotiation.cc:407] Connection header received
0809 04:49:49.579306 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.579309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.579356 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.579405 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.579881 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60848: BlockingRecv error: recv got EOF from 127.0.0.1:60848 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 04:49:49.585541 799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.583976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60852 (local address 127.7.171.61:44703)
0809 04:49:49.584127 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.584131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.584647 (+ 516us) server_negotiation.cc:407] Connection header received
0809 04:49:49.584834 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.584838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.584887 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.584937 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.585435 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60852: BlockingRecv error: recv got EOF from 127.0.0.1:60852 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 04:49:49.593366 799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.591655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60858 (local address 127.7.171.61:44703)
0809 04:49:49.591814 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.591818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.592399 (+ 581us) server_negotiation.cc:407] Connection header received
0809 04:49:49.592600 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.592604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.592656 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.592712 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.593252 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60858: BlockingRecv error: recv got EOF from 127.0.0.1:60858 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260809 04:49:49.605377 799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.603626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60868 (local address 127.7.171.61:44703)
0809 04:49:49.603747 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.603752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.604371 (+ 619us) server_negotiation.cc:407] Connection header received
0809 04:49:49.604576 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.604580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.604635 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.604689 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.605255 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60868: BlockingRecv error: recv got EOF from 127.0.0.1:60868 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 04:49:49.625787 799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.623904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60874 (local address 127.7.171.61:44703)
0809 04:49:49.624035 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.624040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.624720 (+ 680us) server_negotiation.cc:407] Connection header received
0809 04:49:49.624942 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.624946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.625004 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.625061 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.625658 (+ 597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60874: BlockingRecv error: recv got EOF from 127.0.0.1:60874 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 04:49:49.662089 799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.660654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60886 (local address 127.7.171.61:44703)
0809 04:49:49.660787 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.660793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.661063 (+ 270us) server_negotiation.cc:407] Connection header received
0809 04:49:49.661220 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.661224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.661288 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.661408 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.661957 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60886: BlockingRecv error: recv got EOF from 127.0.0.1:60886 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 04:49:49.729943 799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.728784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60890 (local address 127.7.171.61:44703)
0809 04:49:49.728938 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.728941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.729093 (+ 152us) server_negotiation.cc:407] Connection header received
0809 04:49:49.729212 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.729214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.729277 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.729365 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.729839 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60890: BlockingRecv error: recv got EOF from 127.0.0.1:60890 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 04:49:49.862035 799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:49.860640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60896 (local address 127.7.171.61:44703)
0809 04:49:49.860853 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:49.860858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:49.861049 (+ 191us) server_negotiation.cc:407] Connection header received
0809 04:49:49.861173 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:49.861175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:49.861238 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:49.861320 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:49.861884 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60896: BlockingRecv error: recv got EOF from 127.0.0.1:60896 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260809 04:49:50.121604 799 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:50.120453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60904 (local address 127.7.171.61:44703)
0809 04:49:50.120595 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:50.120599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:50.120808 (+ 209us) server_negotiation.cc:407] Connection header received
0809 04:49:50.120930 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:50.120933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:50.120984 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:50.121070 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:50.121485 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60904: BlockingRecv error: recv got EOF from 127.0.0.1:60904 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 04:49:50.423425 912 catalog_manager.cc:1259] Loaded cluster ID: 79d5b6329d0b40e6ace08d4ae90b27b3
I20260809 04:49:50.423513 912 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387: loading cluster ID for follower catalog manager: success
I20260809 04:49:50.425308 912 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387: acquiring CA information for follower catalog manager: success
I20260809 04:49:50.426493 912 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 930dcee525574bcabd0b78d85116b387: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:49:50.477003 935 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a: acquiring CA information for follower catalog manager: success
I20260809 04:49:50.478192 935 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e929284c181946369435679bf0db5b4a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:49:50.640169 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:50.635855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.7.171.61:44703)
0809 04:49:50.636143 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:50.636148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:50.636305 (+ 157us) server_negotiation.cc:407] Connection header received
0809 04:49:50.636482 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:50.636490 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:50.636625 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:50.636745 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:50.637021 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:50.637602 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:50.638521 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:50.638850 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:50.638960 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:50.639293 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:50.639732 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:50.639870 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:50.639912 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 04:49:50.639967 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":97,"threads_started":1}
I20260809 04:49:50.646960 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:50.643099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.7.171.60:45319)
0809 04:49:50.643364 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:50.643370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:50.643385 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:49:50.643584 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:50.643587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:50.643732 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:50.643857 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:50.644097 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:50.644621 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:50.645473 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:50.645714 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:50.645819 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:50.646129 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:50.646540 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:50.646697 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:50.646743 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 04:49:50.646800 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":71,"threads_started":1}
I20260809 04:49:50.653855 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:50.649906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59056 (local address 127.7.171.60:45319)
0809 04:49:50.650017 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:50.650021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:50.650096 (+ 75us) server_negotiation.cc:407] Connection header received
0809 04:49:50.650254 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:50.650260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:50.650319 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:50.650392 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:50.650726 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:50.651473 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:50.652544 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:50.652776 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:50.652878 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:50.653205 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:50.653280 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:50.653339 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:50.653668 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 04:49:50.653721 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:49:50.654332 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:50.650465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50168 (local address 127.7.171.61:44703)
0809 04:49:50.650586 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:50.650591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:50.650918 (+ 327us) server_negotiation.cc:407] Connection header received
0809 04:49:50.650975 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:50.650978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:50.651033 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:50.651101 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:50.651353 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:50.652117 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:50.653197 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:50.653437 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:50.653546 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:50.653815 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:50.653908 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:50.653977 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:50.654154 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 04:49:50.654215 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:49:50.655073 1028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:50.649889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34652 (local address 127.7.171.62:46667)
0809 04:49:50.650159 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:50.650164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:50.650462 (+ 298us) server_negotiation.cc:407] Connection header received
0809 04:49:50.650569 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:50.650572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:50.650767 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:50.650880 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:50.651231 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:50.651873 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:50.653139 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:50.653479 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:50.653597 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:50.654242 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:50.654744 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:50.654838 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:50.654891 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 04:49:50.654954 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":69,"threads_started":1}
I20260809 04:49:50.657891 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 642
I20260809 04:49:50.667954 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 720
I20260809 04:49:50.676687 7852 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfTSqH0/build/debug/bin/kudu with pid 801
2026-08-09T04:49:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3262 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6563 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (293053 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST